Subject: Re: 3. Requirements - Anonimity (was Re: FW: [Asrg] 0. General)

From: "Alan DeKok" <[email-masked]>
Date: Sun, 26 Oct 2003 17:22:18 -0500

Brad Knowles <[email-masked]> wrote:
> 	If you don't want to be held accountable for telling the world 
> about SARS II, you can choose not to speak.
> 
> 	Sounds like a solution the Chinese government would love.

  Nice.  Can we bring the Nazi's into the conversation, too?

> 	How many recipients in this world have total control over the 
> mail system that they use?  If the operator of their mail system 
> chooses to filter all their mail through the Chinese government, what 
> choice do they have?

  Telephones?  Radios?  Carrier pigeons?  Couriers?  Letters?  Word of
mouth?  Stegonography?  Webmail?  SUBMIT?  FTP?  Bi-directional POP?
IMAP?  Chat?  Web discussion forums?  Instant Messaging?  The list can
go on ...

  Is it really that difficult to figure out?

  You are operating under the implicit assumption that all of the
worlds problems must be solved via SMTP.  I fail to see why.  The
world is much bigger than SMTP.

  I feel like I'm seriously missing something in this conversation,
like a hidden agenda.  I hate it when that happens.

  Alan DeKok.

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: "Jon Kyme" <[email-masked]>
Date: Mon, 27 Oct 2003 09:14:35 +0000

Either
a. We're challenging the anonymity section in the requirements document.
- I guess if there's a consensus that this isn't a requirement it would be
struck out.

Or
b. We go with the requirements in this matter and accept that all proposals
must
(explicitly) "address issues of accountability and anonymity" - a
paradoxical but key assertion.

--

---

From: "Alan DeKok" <[email-masked]>
Date: Mon, 27 Oct 2003 09:33:39 -0500

> - I guess if there's a consensus that this isn't a requirement it would be

  Anonymity is unrelated to many of the anti-spam systems.  As PHB
pointed out, holding an ISP accountable for a message means that a
user of that ISP can still be anonymous.

  There is no requirement that the user ID at that ISP map to a real
person, in a well-known fashion.  There is no requirement that the ISP
know a users real identity, and not accept cash transactions from
anonymous people off of the street.  In places where people CANNOT be
anonymous to an ISP, they CANNOT use SMTP to create anonymity where
none existed before.  They MUST use other methods to send anonymous
messages.

  Having a requirement that SMTP creates anonymity in hostile
situations is a simple way to guarantee that every anti-spam proposal
will fail to satisfy that requirement, and will therefore be rejected.

  Alan DeKok.

---

From: "Eric Dean" <[email-masked]>
Date: Mon, 27 Oct 2003 10:06:16 -0500

>   Anonymity is unrelated to many of the anti-spam systems.  
> As PHB pointed out, holding an ISP accountable for a message 
> means that a user of that ISP can still be anonymous.

Can an ISP be held accountable for a message and still ensure privacy
and free-speech?  On a more technical level, should an ISP inspect every
outgoing message and ensure that it's not spam?  Is the anti-spam war
more easily fought at the source than at the sink?

Rather than just make this rhetorical, there are two types of ISPs:
A) Consumer access: Dial, DSL, Cable, Lease line
B) Content access: Hosting, DS3s...

Regarding consumer access, most ISPs place port 25 filters on their
access lines.  A sender can only send email to a subset of email relays.
This works moderately well.  Most spammers have many alternatives for
sending email (see item B) rather than via the ISPs mail server
specified within the port 25 access list.

Regarding content access, datacenters essentially do next to nothing to
restrict spammers.  In many cases, I receive UBE notices from ISP
providers where the spam originated from their own datacenter.
Considering that datacenters make money on Mbps...they are not concerned
with the excessive traffic generated by spammers.  Some datacenters are
essentially spam magnets.  In these cases various relay DBs attempt to
find these sources and block them...but the effort is futile.

I am still amazed that lawsuits go after the companies generating the
spam rather than the datacenters that turn a blind-eye towards them and
provide all the infrastructure to accommodate.

---

From: "Jon Kyme" <[email-masked]>
Date: Mon, 27 Oct 2003 15:59:14 +0000

> will fail to satisfy that requirement, and will therefore be rejected.

No such requirement is made. Only that proposals address the issue.
If you have strong objections to anything in the requirements document - it
would be best if you made this clear.

Being damaging to anonymity would not be a fatal objection to a proposal.
That a proposal ignores the issue would (IMO) be.

--

---

From: "Jonathan A. Zdziarski" <[email-masked]>
Date: Mon, 27 Oct 2003 11:07:28 -0500

> That a proposal ignores the issue would (IMO) be.

This eliminates all statistical-based filters (Bayesian, Chi-Square,
etc.), which are among the most successful spam tools that exist today,
as the sender's true identity is of no importance to the filter.  These
filters are based on the philosophy that all email will continue to be
anonymous and therefore pays attention to the one tell-tale
characteristic of all spams:  their content.  So I would recommend if
you're going to require that a proposal address the issue of anonymity,
you give the author the ability to suggest that it is ignored by the
filter.

---

From: "Alan DeKok" <[email-masked]>
Date: Mon, 27 Oct 2003 11:52:46 -0500

> No such requirement is made. Only that proposals address the issue.

  Recent discussions on the list leaned towards adding that
requirement.

> would be best if you made this clear.

  My objections were to additional requirements outside of the
document, not what was in the document.

  Alan DeKok.

---

From: David Maxwell <[email-masked]>
Date: Mon, 27 Oct 2003 13:40:20 -0500

On Mon, Oct 27, 2003 at 11:07:28AM -0500, Jonathan A. Zdziarski wrote:
> as the sender's true identity is of no importance to the filter. 

That is because they require no infrastructure. Additionally, you need
to define 'success'. Configuring your MTA to only accept connections
from hosts which are an MX for the domain they HELO as, will also block
lots of spam - it may however have a higher false positive rate.

> These
> anonymous

I disagree. I would say that these filters are based on the issue that
there is no current alternative to anonymous mail, so they address the
only aspect that they can - content.

I don't think it's reasonable to read anything into the philosophy of
filter authors about their expectations for the future - they're just
trying to solve a problem with the available inputs.

> and therefore pays attention to the one tell-tale

Filters certainly don't affect anonymity, but until a program passes the
Turing Test, I won't be convinced that filters can't be fooled.

Even then, textual analysis requires far more cpu overhead than an
access list check. If the spammers have any hope that their message will
get through to you, they have incentive to send more and more spam. If
the spammers have NO hope that their message will get through to you,
they do not have incentive any more.

-- 
David Maxwell, [email-masked][email-masked] -->
Net Musing #5: Redundancy in a network doesn't mean two of everything and
half the staff to run it.
					      - Tomas T. Peiser, CET

---

From: "Jonathan A. Zdziarski" <[email-masked]>
Date: Mon, 27 Oct 2003 13:45:30 -0500

> lots of spam - it may however have a higher false positive rate.

But doing this in conjunction with a statistical filter will weaken the
statistics, because it won't have as much spam to learn.  On the other
hand, if you were to do this, and pass the message "as spam" to the
filter, this would be acceptable.

> only aspect that they can - content.

That's not how most of us feel who are developing these tools; I've
discussed this at length with many of the developers on these projects,
and whether or not the message can be authenticated is irrelevant to the
operation of the filter.  Some spammers are now digitally signing their
messages to appear authentic (some signatures are obviously bogus)...but
should we have an authenticated system of SMTP, that will not affect the
filter's basis premise that it is the _content_ of the message that
makes it guilty, not the sender.  In other words, even if everyone was
using an authenticated version of SMTP, it would not suggest that the
filter should "trust" the sender/message.

> trying to solve a problem with the available inputs.

Philosophy is one of the significant factors that we discuss, actually. 
Philosophy and mathematics.

> Turing Test, I won't be convinced that filters can't be fooled.

No filter is foolproof, but these filters are by far the most accurate
filters available today.  The sender really is irrelevant...it could be
a friend of yours who forwards a message to you they thought was
important, but was really spam...it could be a mailing list you are
subscribed to with one malicious subscriber, etc.

> they do not have incentive any more.

Which is why spammers are using stolen dialup accounts and such to
circumvent access lists...just as they will an authenticated method of
SMTP to send them...but one thing remains the same: the content.

---

From: "Jonathan A. Zdziarski" <[email-masked]>
Date: Mon, 27 Oct 2003 13:51:14 -0500

FYI if this helps, Paul Graham - a member of the spamfilt forum, and one
of the people responsible for the annual Spam Conference at MIT - has
written several articles outlining the basic theory, philosophy, and
design of statistical-based filters.  His website is
http://www.paulgraham.com.  I think you'll find it very useful
information, with a lot of data to back up his theory.

---

From: David Maxwell <[email-masked]>
Date: Mon, 27 Oct 2003 14:28:13 -0500

> filter, this would be acceptable.

This is diverging from the question of anonymity, and I'd rather stick
to the one issue.

> operation of the filter.

That's a different subject. "Would filters still be useful if mail was
authenticated?" - Sure.

> Some spammers are now digitally signing their

Contents which try to look like authenticated mail have nothing to do
with authentication of mail. While useful to discuss in a filtering
context, either an authentication system is breakable, or it isn't - if
it isn't, either a message passes, or it doesn't.

> Philosophy and mathematics.

You seem to be reading my text as "Filter developers don't have a
philosophy", rather than my intended "Don't make guesses about filter
developers' philosophy based on the environment they are working
within."

The later is akin to saying "Anyone who works at a desk philosophically
believes that going outdoors is wrong." (i.e. nonesense)

> information, with a lot of data to back up his theory.     

I searched Paul's site for 'anonymity' and 'anonymous'. Nothing related
to email was found - only one reference to 'anonymous functions', in an
article about programming.

-- 
David Maxwell, [email-masked][email-masked] -->
Any sufficiently advanced Common Sense will seem like magic... 
					      - me

---

From: "Jonathan A. Zdziarski" <[email-masked]>
Date: Mon, 27 Oct 2003 14:28:45 -0500

My point to all of this is that the sender's identity plays no role with
statistical-based filters, other than treating the From field as another
token in the mix...therefore ignoring the sender's identity in the
scheme of anonymity should be an acceptable part of this spec, otherwise
you're eliminating the most effective filters available today.

> developers' philosophy based on the environment they are working

You do realize you're telling a filter developer not to make guesses
about filter developers, right? =)

> article about programming.

Paul discusses the importance of content and the unimportance of
anything else (such as the sender), whitelists, etc.  That's kind of my
point - anonymity is irrelevant to these filters...which is why they
didn't break when sitefinder was turned on briefly, causing all domains
to return as "existing"

---

From: David Maxwell <[email-masked]>
Date: Mon, 27 Oct 2003 14:44:32 -0500

> you're eliminating the most effective filters available today.

I don't want to make filters unworkable - but I wish they were
unneccessary though.

> about filter developers, right? =)

Without the presence of some 'filter developer union' organization, and
its statment "We expect email will always be anonymous", any statement
by an individual can't be taken as the point of view of the 'industry'
as a whole. Many people can do the same thing (write filter systems) and
still hold different beliefs and philosophies.

> didn't break when sitefinder was turned on briefly, causing all domains

So, I'd conclude:

Filters -> No dependence on anonymity (presence or lack of)

That doesn't seem to support any form of statment about filter writers
expectations for the future of email - only about their choice to design
tools which are independent of that future.

-- 
David Maxwell, [email-masked][email-masked] --> The only difference I see
between voodoo and marketing research is that voodoo sometimes works! 
						- Leonard Stern

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 27 Oct 2003 14:47:04 -0500

> you give the author the ability to suggest that it is ignored by the

Nothing is cast in stone - the proposal author will point out whether 
specific requirements apply to his proposal or not. For example, 
creative addressing proposal that was discussed a while back has an 
analysis done by the author based on both the requirements document and 
the technical considerations document. Whatever did not apply, was 
indicated so.

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 27 Oct 2003 14:48:38 -0500

> specified within the port 25 access list.

However, if restricted by alternatives will spammers try to go through 
the ISP and will that lead to the ISP cutting down on anonymity?

---

From: "Jonathan A. Zdziarski" <[email-masked]>
Date: Mon, 27 Oct 2003 14:50:14 -0500

> I don't want to make filters unworkable - but I wish they were

If you're expecting authentication to obsolete filters, you've got quite
a surprise coming =) Like I said before, spammers will only find a way
to operate within a "trusted" environment (which as long as it's public,
will never truly be trusted).  As long as domains are $8.95, this will
never happen.

> still hold different beliefs and philosophies.

1. There is a rather large group of filter developers, called the
spamfilt group.  It is a private list, however, just for developers
which is probably why you may not have heard of it.  This is how I was
able to craft an internet draft with the author of CRM114, along with
the input from several other filter developers such as those from
Bogofilter and SpamProbe.

2. The _foundation_ of statistical-based filters is that the real
pudding is in the content, and not the sender.  They are designed by
nature to operate with the philosophy that headers are forgeable and all
mail is "untrusted"...even if you apply some type of semi-trusted
environment for email, these filters will continue to look at all mail
as untrusted.

> tools which are independent of that future.

Real Paul's explanation of "the emails of the future".  It is based
around content, not your incorrect belief that an authenticated method
of mail exchange will affect spam in any way.

It's very easy to sit here and criticize filters for ignoring things
that we think are important, but I would rather focus my attention on
the individuals who are actually coding these spam filters as they [we]
are far more familiar with what works and what doesn't, what will work
and what will not, and why the things that you think are important may
not actually be...rather than sitting and talking about theory, lets
take a look at the people who are actually accomplishing something at
fighting spam on the front-lines.  In other words, it is the filters
that are already succeeding that validate your requirements doc....it's
not your requirements doc that validates the spam tools.

Jonathan

---

From: "Jonathan A. Zdziarski" <[email-masked]>
Date: Mon, 27 Oct 2003 14:51:43 -0500

> specific requirements apply to his proposal or not. 

That's the general point I was hoping everyone could agree on...if they
don't hold anonymity to be relevant to their proposal, then they should
be able to have their proposal considered with that statement, rather
than be rejected for not being relevant to anonymity =)

---

From: "Jonathan A. Zdziarski" <[email-masked]>
Date: Mon, 27 Oct 2003 14:53:56 -0500

> the ISP and will that lead to the ISP cutting down on anonymity?

Being that a majority of spammers who use ISP accounts are either using
stolen accounts, or accounts they purchased with a pre-paid credit card,
I submit that even if the ISP cracks down on anonymity and is able to
identify the user on their system, the spammer will still be able to
maintain complete anonymity as a result of their utilization of the
account.

So then we're back to square one: we've authenticated a user at an ISP
who doesn't really exist in real life =)

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 27 Oct 2003 15:02:55 -0500

> of mail exchange will affect spam in any way.

"Email + authentication = no spam" has been a common approach by many. 
Dave Crocker, Vernon Schryver and many others have pointed out that this 
particular equation does not compute just like SMTP-AUTH did not kill spam.

However, that does not mean that filters is the only way to go. As per 
the ASRG charter, recasting everything as "consent-based communications" 
and providing protocols to unify various anti-spam tools including 
filters may be something that we want to concentrate on instead.

> not your requirements doc that validates the spam tools.

Going off-tangent for a second - is there anything for filter developers 
that the ASRG or IRTF/IETF can do in terms of perhaps helping to develop 
a common standard of some sort?

Yakov

---

From: "Jonathan A. Zdziarski" <[email-masked]>
Date: Mon, 27 Oct 2003 15:07:46 -0500

> a common standard of some sort?

Absolutely, this is kind of where my private email to you was going...so
I'll copy part of it to the list.  The IRTF would be much more useful to
the anti-spam community if rather than trying to define how chickens
should lay eggs, if their goal was to observe how chickens did in fact
lay eggs, and document a "best practices doc" to help define standards
for other chickens that might be having difficulty with their eggs.

Translation: There are a lot of very talented individuals in the
anti-spam community (many of which are exponentially smarter than I am)
who have succeeded in filtering 99.9% of all spam (we're all hovering
around three-nines accuracy right now)...and made their research
available to the world.   If this research and input were compiled into
your requirements documentation, then you would be creating a document
that defines standards based on actual results instead of theory.  This,
IMHO, would make a significantly more useful document to benefit the
networking community.

Jonathan

---

From: David Maxwell <[email-masked]>
Date: Mon, 27 Oct 2003 15:20:01 -0500

> will never truly be trusted).  As long as domains are $8.95, this will

You're making assumptions about what types of authentication I have in
mind. One of your assumptions seems to be about domain-based
blacklisting. That's not a method I consider particularly worthwhile
(but that's just my opinion).

> Bogofilter and SpamProbe.

The existence of, and your participation in the group, do not turn your
opinion into a statement for the group, unless they've made you an
official spokesperson.

> of mail exchange will affect spam in any way.

When quoting an article, please include the exact title, and a URL. I'd
be happy to read it so that we have more common ground to discuss from,
but I can't find anything called that on Paul's site.

As for my 'incorrect belief', for a trivial example, if I set up a
mailbox which discards all mail except for that which is PGP signed by a
particular contact of mine, then my consent policy of "Accept mail from
Joe", is perfectly implemented. (As long as Joe will sign all his
messages, for my convienience.)

If the SMTP protocol 'told' connections that this was going to be done,
then spammers would learn that sending messages to that mailbox was a
waste of time.

For another comparison, why do banks control access to funds with a
signature, instead of saying 'Is this the type of phone bill that David
would actually want to pay?'

Content Filters have their place. Access controls have their place.
Neither is a subset of the other. Neither can replace all the
functionality of the other. Access controls have substantially less
overhead, which means that in a well designed system, it should be
possible for people to choose to use them first, to save the resources
needed for Content Filtering.

Additionally, Access controls are a 'hard stop', while Content Filters
are a 'soft stop'.

Like the virus/anti-virus continual escalation, spammers can write
'smarter' spam to bypass the newest filters.

In the case of Access controls, if someone has a tight consent
definition such as 'I accept mail from these four people', and if mail
senders are authenticated, then this recipient will _never_ get
something outside of that consent definition.
 
> It's very easy to sit here and criticize filters for ignoring things

I don't recall making any criticism of filters besides a) cpu intensive
and b) can't ever be perfect.

> In other words, it is the filters

Those filters are doing a good job, given the current (Aug 1982 for
RFC822, April 2001 for the cleanup in 2822) SMTP environment, but that
doesn't mean that the environment can't be improved.

Put another way, if SMTP doesn't start to support (or this group make
progress towards upgrading it to support) consent policies based on
authenticated transactions, then I suspect large numbers of people will
start using something else for messaging.

-- 
David Maxwell, [email-masked][email-masked] --> Mastery of UNIX, like
mastery of language, offers real freedom. The price of freedom is always dear,
but there's no substitute. Personally, I'd rather pay for my freedom than live
in a bitmapped, pop-up-happy dungeon like NT. - Thomas Scoville

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 27 Oct 2003 15:25:47 -0500

> On Mon, Oct 27, 2003 at 02:50:14PM -0500, Jonathan A. Zdziarski wrote:
<snip>
> something outside of that consent definition.
<snip>

This is leading back towards the consent framework and consent. Our 
charter calls for consent-based communications where a "super-system" is 
defined where all various anti-spam tools are tied in together and users 
can choose the tools they want to use.

---

From: Brad Knowles <[email-masked]>
Date: Tue, 28 Oct 2003 04:06:34 +0800

>  filter should "trust" the sender/message.

	Thank you!

	It doesn't matter whether they're using open proxies, trojaned 
machines, stolen accounts, accounts obtained under false pretenses 
(e.g., with fake credit cards), or any other technique currently 
known today or not, there is an increasingly large number of spammers 
who are perfectly willing to use "authenticated" methods to transmit 
their spam.

	Even if we forced everyone in the universe to send e-mail using 
only the authorized outbound mail relays from their provider, using 
only the authorized domains owned by their provider, we would not 
stop this onslaught.  Moreover, I don't think we'd even slow it down 
very much.

	Providers would tend to turn a blind eye to this sort of thing 
because they don't want to build the kind of mail system 
infrastructure that could deal with those loads, and they don't have 
the wherewhithal to build an individual user authentication method 
that could reliably tell the difference between legitimate and 
illegitimate uses of the mail system -- if your machine is trojaned 
and using your MUA of choice to send out e-mail through your 
designated outbound mail relays, how is the provider going to tell 
the difference between that illegitimate use and your real-world 
outbound messages?

	Are we going to ask them to filter all the mail through the 
Chinese gov't or the NSA, so that we can get back an "authoritative" 
answer as to whether or not this message is okay?  If not, how can 
any one ISP hope to put together the kind of resources it would take 
to make those kinds of determinations based on client patterns, and 
which could notice "abnormal" behaviour (i.e., six-sigma 
differences)?  And if they did, why shouldn't we expect spammers to 
use a "slow start" mechanism that would fly under the radar?  And if 
slow start wasn't enough, what if the patterns would only show 
themselves when data is collected across all ISPs?  Shall we have all 
e-mail messages and log data transmitted in real-time to the 
Department of Homeland Security, the Mossad, or maybe Jemaah 
Islamiyah?

	The worst part of it is, that as we throw out one piece of the 
baby with the bathwater at a time, soon enough there won't be any 
baby left.  Even if it's just one blood cell at a time, if you do it 
enough, the result is that there is nothing left.  This is the 
reverse of the "one bite of the apple" proposal that DMA spokesperson 
Patricia Faley told CNN, and which is debunked at 
<http://www.cauce.org/pressreleases/math.shtml>.  This time, we'd be 
doing it to ourselves, while not stopping (or even significantly 
slowing) the spammers.

	I refer you to the quote in my .sig.  The people proposing these 
non-crypto based "authentication" mechanisms are suggesting that we 
each give up essential e-mail liberties one by one for very little 
temporary safety that will have no positive impact in the long run.

	We need totally different kinds of solutions to this problem.

	For one, if you're going to do authentication, don't bother 
trying to do anything through the DNS -- That's a thermo-nuclear 
minefield just waiting to happen as it is.  Unless you are Cricket 
Liu, Paul Vixie, or some other DNS luminary that could write their 
own book on DNS Security [*], then you are not qualified to judge the 
risks of this approach.  If you're going to do authentication, then 
do it right -- using a secure protocol with cryptographic techniques 
at the core.

	Secondly, if you insist on going down this path despite all 
warnings to the contrary, don't be silly and insist that this is 
something that can be done in a binary manner -- that might work in 
an ideal world, but this has never been an ideal world.  Instead, use 
this information as just one input to a overall scoring system that 
can take many factors into consideration, and then try to come to a 
more intelligent decision based on all the available data.

[*]  If you have already done so, or are in the process, I want to talk to you.

-- 
Brad Knowles, <[email-masked]>

"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
     -Benjamin Franklin, Historical Review of Pennsylvania.

GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)

---

From: "Jonathan A. Zdziarski" <[email-masked]>
Date: Mon, 27 Oct 2003 15:37:59 -0500

> opinion into a statement for the group, unless they've made you an

As a researcher, it should not be considered good practice to ignore
data as you have in this case.  Simply denying the usefulness of
information because you happen to disagree with it is disturbing, as you
appear to be trying to tell the rest of the world what a spam filter
should and shouldn't be, when the developers have already figured that
out for you.  IMHO, if you're going to write up a requirements doc
after-the-fact (as you are here), you ought to be playing scribe and
project manager, not scientist.

> messages, for my convienience.)

This is far from perfect.  What if Joe's laptop gets stolen?  What if
someone intercepts your key exchange and the PGP signature that gets
delivered to you is actually from Mallory?  What if Joe's machine gets a
virus that allows either someone access to it, or someone to get his
keys from it?  There is no guarantee that Joe is Joe, even with
PGP....which is why if you all of a sudden start getting spams from Joe,
you'll get the idea that Joe has been compromised...therefore you can
never "trust" Joe.

> then spammers would learn that sending messages to that mailbox was a

On the contrary, spammers would use either one of the above methods to
hijack authenticated users, or become an authenticated user themselves -
and if you expect any type of authentication to be accepted publicly, it
will need to be easy enough to become authenticated so as it doesn't get
in the way of turning a profit - meaning spammers will be able to turn a
new "identity' every day...regardless of whether it's got to do with
domains, downstream users, or what-have-you.

> would actually want to pay?'

And as a result checkbooks still get stolen on a daily basis, and there
are enough phony credit card transactions to keep the secret service
busy for the next decade.

If, however, the bank did look at behavioral patterns (some banks do)
then they would be able to contact you when they see a discrepancy.

> Neither is a subset of the other.

Applying access controls in front of content-filers actually weakens a
content-filter, so one might say the only way to make them cohabitate is
to get them to talk to each other.

> needed for Content Filtering.

It depends what kind of door you want.  If you want a wooden door or a
steel door, and how much of a trade-off you're willing to make between
resources and accuracy.  You have to measure the resources you would use
in handling spam abuse reports, retraining, and other functions that
occur as a result of receiving one spam...then you have to compare that
to the additional resources it would take to perform content-filtering. 
If content-filtering requires more resources than processing the spams
that will get through your front-line access control security, one must
conclude that content-filtering is not an appropriate method...however
in most cases, I think you'll find that the amount of spam that gets
past most front-line access controls is significant enough to warrant a
content-based filtering solution.  

You also need to take into consideration the 'usability' issue with
regards to access lists.  If you are using a reply-based list (where you
have to reply to get added to their access list), there are many
individuals (including myself) who _will not_ converse with someone
using this approach for many reasons including both resources and
philosophy.  If you want to catch 100% of spams, disconnect your mail
server from the public Internet - but you'll find it's not quite as
usable.

So there's a balance...I'm not saying access controls are useless...but
they certainly are no single solution.

> Additionally, Access controls are a 'hard stop', while Content Filters

However, I'd rather have a soft-stop of 99.9% of spams than a hard-stop
of 70% of spams.  That's up to the admin though to decide.  Several
"soft" solutions are designed to feed information to the "hard"
solutions to get the benefits of both types of filters.

> 'smarter' spam to bypass the newest filters.

So far, this hasn't been the case.  Good statistical-based filters learn
these new patterns, and accuracy is only getting better - not worse. 
This is my argument, however, against access controls - content-based
filters learn, whereas access controls do not...they are a standardized
environment spammers can learn to function in.  A spam getting through
the spammer's own test filters is no guarantee that the spam will make
it past my filters, but a spam getting through the spammer's test lab
using the same access controls as your access-based filter is going to
get through any identical filter.

> something outside of that consent definition.

Statistical-based filters do experience similar behavior, although it's
all a wash in the mix.  Tokens from the sender such as their return
address, mail user agent, signature, etcetera all come into play in a
final calculation.  Although unintentional, whitelists are a common
result of statistical-based filters...even if my argument is that they
don't need to be.

> a) cpu intensive

This is a very relative assumption.  How would you define CPU
intensive?  DSPAM is written entirely in C and actually uses fewer
resources than many access-based filters written either in PERL, or
poorly-written in C/C++.

> and b) can't ever be perfect.

Nothing can ever be perfect, though, including access-based filters. 
Ultimately any requirements doc should be able to support both types of
filters...is all my argument ever was....certainly not a flame of
access-based filters.

> start using something else for messaging.

My belief is that a mail server should deliver mail - just as a router
routes packets.  Leave it to bolt-on solutions to manage
consent-policies.  Even if you write the best consent policy requirement
in the world, it's not going to win over a majority of users because
admins want to run what works for them.  This doesn't belong in a
standard for SMTP, IMO.

---

From: "Jonathan A. Zdziarski" <[email-masked]>
Date: Mon, 27 Oct 2003 15:49:06 -0500

> can choose the tools they want to use.

That is a worthy cause =)

Have we taken into account that many anti-spam tools do not necessarily
instantiate at MTA time, but MUA time?  Does the scope of this framework
cover that, or is it specifically for MTA spam tools?

---

From: "Alan DeKok" <[email-masked]>
Date: Mon, 27 Oct 2003 16:25:16 -0500

Brad Knowles <[email-masked]> wrote:
> answer as to whether or not this message is okay? ...  Shall we have
> all e-mail messages and log data transmitted in real-time to the

  Do you have anything positive to offer?  The only thing I can
determine from your posts is that you are opposed principle to any
attempt to modify SMTP.

  I think we can agree that you think SMTP was handed down from on
high in the form of stone tablets, and that we're all going to h*ll
for trying to change it.

  That being said, do you have *anything* positive to say, which will
help address the spam problem, and not result in eternal damnation?
Is there ANY possibility that an anti-spam proposal will someday
satisfy your standards?

  Alan DeKok.

---

From: Dennis Gearon <[email-masked]>
Date: Mon, 27 Oct 2003 13:34:42 -0800

>start using something else for messaging.
I don't think authentication will work, unless it's iron clad, like, 
"This call is coming from a pacific bell account". The hardware is 
reporting the pretty much undeniable reality of a call.

What I think you're going to see is user group web sites, and the demise 
of regular email, if it doesn't adapt soon. To 'email' someone, you will 
have to be on a website they are on, be a member, AND have permission to 
talk to them. Your simple permission or security token will have to be 
exchanged OUT of bounds from the system.

As an example,
I don't get any BS or spam from:

    My ICQ account
    My AIM account
    My YIM account

via chat methods. Spammers ARE working those protocols, sites, and 
accounts. I know because, unfortunately, the intial setup of all three 
of those allows at least a spammer to ASK you if you want to be talked 
to. I got a LOT of those for porn SPAM until I found the right settings. 
So, there is:
   
    a working example of,
    a ONE HUNDRED PERCENT EFFICIENT,
    current,
    in production,
    in the real world,
    permission based system,

That blocks SPAM.

That these systems work, is irrefutable. The only difference between 
those three systems and RFC email is that they operate off of central 
servers, instead of cooperating, distrubuted servers.

No content filtering required.

---

From: "Jonathan A. Zdziarski" <[email-masked]>
Date: Mon, 27 Oct 2003 16:46:36 -0500

> I know because, unfortunately, the intial setup of all three 

Permission-Based Systems running on top of email have been widely
rejected for many years now for several issues including resources and
time.  There are still a few ISPs that use them, but only a very small
percentage...they're troublesome, waste a ton of bandwidth, and waste a
lot of time.

But the design is perfect, because it requires both an ACK from the
sender and an approval from the recipient...I for one would rather
receive spams than have to deal with this myself, and I'm a savvy
person..think of all the grandma's out there who wouldn't be able to use
email because they couldn't figure the system out.

---

From: "Jon Kyme" <[email-masked]>
Date: Mon, 27 Oct 2003 22:49:04 +0000

> than be rejected for not being relevant to anonymity =)

Surely. If there's something in our requirements that one doesn't believe
apply to a proposal (whatever it is) it's probably best to make this
explicit from the start. However, it's got to be a 'legitimate' objection
to say that this is wrong and the requirement does / must apply to a
proposal and that the proposer should go away and consider it.

Also, we can argue about the requirement(s) - rather than for or against
some particular proposal. This should save us having to reprise all the
arguments for each proposal.

In this case, do we believe that anonymity is:
(a) a property of the current MTS 
(b) desirable
(c) worth providing for at some cost in anti-spam effectiveness
    (or accountability)

Is there a broad consensus on this? Do we need a consensus on this?
Me, I'm not sure about 'a'.

--

---

From: Markus Stumpf <[email-masked]>
Date: Mon, 27 Oct 2003 23:59:07 +0100

> exchanged OUT of bounds from the system.

Imagine you run a business.
Would you adopt your info@ account to the above mentioned system?
How much new customers do you think you would get compared to another guy
in the same business going through the hassle of an "open" info@ account
and 300 spam mails a day?

	\Maex

-- 
SpaceNet AG            | Joseph-Dollinger-Bogen 14 | Fon: +49 (89) 32356-0
Research & Development |       D-80807 Muenchen    | Fax: +49 (89) 32356-299
"The security, stability and reliability of a computer system is reciprocally
 proportional to the amount of vacuity between the ears of the admin"

---

From: "Jon Kyme" <[email-masked]>
Date: Mon, 27 Oct 2003 23:04:23 +0000

>admins want to run what works for them.  This doesn't belong in a

My MTA enforces tons of policy about what mail it should transport. This
seems perfectly sensible - it doesn't promiscuously transport all mail to
anywhere. I'd like my users to have their individual policies enforced at
our organisational boundary. Why am I wrong?

--

---

From: "Jonathan A. Zdziarski" <[email-masked]>
Date: Mon, 27 Oct 2003 18:06:48 -0500

> our organisational boundary. Why am I wrong?

If a true consentual message interface were designed, this interface
could be used to enforce all of your current policy...access lists,
relays, etc.

---

From: Dennis Gearon <[email-masked]>
Date: Mon, 27 Oct 2003 15:57:26 -0800

Dennis Gearon wrote:

>>> How about an XML standard that defined ways that a mail server could 
>>> accept/bounce/drop down a canyon/etc emails? The users uploads the 
>>> file and voila, enforcement happens while email client is offline. 
>>> Of course, it needs a nice SIMPLE GUI, etc.
>>>   
>>
>>
>> XML is pretty bandwidth intensive though, as compared to some smaller
>> protocol.  If this were to become a widely accepted standard, an
>> architecture might emerge where you could have 100 "dumb" mail servers,
>> and one or two "policy servers" for lack of a better name.  In that
>> case, you'd want to have the most lightweight protocol for communicating
>> between them.   Maybe XML will work... hmm.
>>  
>>
> This XML file  only gets transmitted when the user changes his/her 
> policy. It resides for the most part, unchanged on the user's email 
> server, telling the mail server the mail user's wishes by 'proxy'.
>
This would be a very easy way to combine:

    white lists
    gray lists
    filtering standards
    DNS based stuff

for the filtering standards, something like this could be done:

<filter application>
    <filter name>Spam Assassin</filter name>
    <min version>4.5.6.7.8.9-super_duper_beta<min version>
    <specific criteria one> likdra;lekfwe;ofjn</specific criteria one>
    <specific criteria two> likdra;lekfwe;ofjn</specific criteria two>
</filter application>

Upon upload, the mail system can reject it saying, 'don't have such 
application', or if the application exists, the application can reject 
the input based on the version number or the 
missing/wrong/inappropriately mixed  criteria, transparently to the MTA. 
It just returns the string.

Or in the cases of point-to-point enforcement:

<filter application>
     <filter name>body_embedded_PKI_digitally_signed_testing 
application</filter name>
     <accept list>
          <accept from>
               <email address>[email-masked]</email_address>
               <name>joe smuckatelly</name>
               <embedded phrase at beginning of body>chicken ranchers 
have lots of chicken doo doo for sale</embedded phrase at beginning of body>
               <encryption method>blah blah</encryption method>
               <base_64_crypto_key>iebt0uq2b350[n9u2   b5p9n   
2b45p'24    b 'u2b45p9   2v459nqwvtu.onjvrlinvwn</base_64_crypto_key>
          </accept from>
          <accept from>
             blah blah blah
          </accept from>
       </accept list>
</filter application>

---

From: Dennis Gearon <[email-masked]>
Date: Mon, 27 Oct 2003 16:02:23 -0800

Jonathan A. Zdziarski wrote:

>Same for system policy I assume?
>
>So you're talking about a standard for writing policy for any MTA is
>this correct?
>
>
>  
>
yes, and filter applications have to accept the contents of their 
sections with possibly tow scenarios:

    1/ Can you enforce this?
    2/ Is this mail accepted?

All ways to prevent reception are based on plugins, each having their 
own section in the document.

The document could even have an 'ordering  section'

Do filter 1 4X per document, then filter 6 1X, then filter 1 again, then 
filters 2,5,4 and 3. If still accepted, store for user.

---

From: Dennis Gearon <[email-masked]>
Date: Mon, 27 Oct 2003 17:05:59 -0800

Jonathan A. Zdziarski wrote:

>>accept/bounce/drop down a canyon/etc emails? The users uploads the file 
>>and voila, enforcement happens while email client is offline. Of course, 
>>it needs a nice SIMPLE GUI, etc.
>>    
>>
>
It could be a 'dot' file on a unix machine which is the mail server.
I don't know what the equivilent is in windows, ( but unfortuately, 
someday I'm sure I'm going to have to learn it :-)

---

From: "Brian Adams" <[email-masked]>
Date: Mon, 27 Oct 2003 19:28:10 -0800

First, I believe that we need some definitions for Anonymity and Accountability.  I see several uses for the term Anonymity when used in this discussion.  Is it Anonymity as it applies to the real life identity of the  Message Originator? Or perhaps Anonymity as it applies to the Message Origination?   

Second, I disagree with the below statement from the requirements document [2.24.1] because the use of the term "sender" as defined in the requirements ("A computer system, application or process that is the origin...") is used when I believe we want to specify the person sending the email.

> Additionally, there is just cause for preserving sender
>anonymity and supporting the use of appropriate 'anonymization'
>services that currently exist.  Designers must consider the
>impact of accountability by a proposal on these systems and 
>address the issues related to preserving anonymity for 
>[legitimate] uses.

Specifically, the term "just cause for preserving sender anonymity".  Shouldn't that be "just cause for preserving message originator anonymity"?  

Rational:

At the end of the day what we want is to be able to provide a means for 'people' to send email while hiding their identity, right?  And as long as ISPs can provide email addresses without verifying the identity of the person provided the address then the need of message originator anonymity has been fulfilled.  Thus any proposal that does not force ISPs to verify the message originator satisfies requirement 2.14.  This includes my personal favorite the RMX type solutions.

---

From: Dennis Gearon <[email-masked]>
Date: Mon, 27 Oct 2003 21:12:54 -0800

>between them.   Maybe XML will work... hmm.
Another way to save on the need to read this file for every mail is to 
have the relevant applications cache this either in memory, or in a 
filetype of their choosing and under their control.

The command set to each application or the API would have a commmand for 
'reload' and give the file location. It would take out what is relevant 
to it and ingnore the rest. I think a GET/POST/argv/argc/command line 
argument approach would work to supply file name.

---

From: David Maxwell <[email-masked]>
Date: Tue, 28 Oct 2003 12:21:46 -0500

> > > specified within the port 25 access list.

(I'd disagree with the statement that 'most ISPs' filter port 25, but
whatever ;-)

> who doesn't really exist in real life =)

The issue here is that accountability facilitates a web of trust (or web
of distrust) - if that ISP is known to give out anonymous accounts, then
a recipient can choose that their 'view of the internet' shouldn't
include receiving mail from users at an ISP with that policy.

This is quite comparable to the phone company offering you the option of
blocking calls from people whose caller-id information is not available.

							David

---

From: "Alan DeKok" <[email-masked]>
Date: Tue, 28 Oct 2003 12:35:50 -0500

> maintain complete anonymity as a result of their utilization of the

  That's fine.

> who doesn't really exist in real life =)

  I don't think so.

  None of the RMX-style solutions authenticate users.  To do so would
be nonsensical.  That's why SMTP AUTH doesn't do anything about spam.
I'm not going to authenticate your users, so it doesn't prevent spam
from reaching me.  You've already authenticated the spammers & agreed
to spamming, so it doesn't prevent spam from leaving your site.

  RMX & friends make ISP's *accountable* for the messages they send.
The importance of accountability versus authentication has been
stressed numerous times on this list.

  With open proxies & relays, neither ISP's nor users are held
accountable for messages, because ISP with the open proxy doesn't
care, and the spam isn't traceable past the open proxy.

  Alan DeKok.

---

From: David Maxwell <[email-masked]>
Date: Tue, 28 Oct 2003 12:52:52 -0500

> project manager, not scientist.

I don't want to ignore the applicability of filters, but you jumped into
the middle of a thread about activists and anonymity and your posts
seemed to be trying to shanghai the thread.

Discussing filters is useful - disrupting a discussion about
accountability with 'but filters are better, because they don't care!'
is simply... disruptive.

> you'll get the idea that Joe has been compromised...therefore you can

Events such as 'Joe's laptop gets stolen' are likely to be quite rare.
As such, the risk of receiving spam due to this situation is so low that
it doesn't present much of an argument to convince me that I should
spend CPU resources doing content based filtering on every mail I
receive (if there are other options for eliminating some quantity of
spam).

Additionally, you highlight the fact that ISPs need tools to provide
better control over users use of resources. If Joe normally sends less
than 20 emails a day, and a spammer takes over his identity, shouldn't
200+ emails in one day be enough to shut him down? 

No, you can never 'trust' anyone, and you can never filter perfectly,
but if each 'identity theft' could only result in a handful of spam
reaching the network, versus the millions delivered today, then the
change would be significantly beneficial to the network as a whole.

> domains, downstream users, or what-have-you.

There are many options here - greylisting would seem applicable, either
on the behalf of the domain-issuing entity or the mail recipient. Weigh
the cost-benefit analysis, do quickly issues domains benefit spammers
exponentially more than regular users? do quickly delivered mail messages
from new communication peers benefit spammers exponentially more than
regular peers?

> then they would be able to contact you when they see a discrepancy.

This isn't comparable to spam. The forged transactions are a tiny
percentage of the valid transactions. Spam on the otherhand, exceeds 50%
of received mail for many users, and 95%+ for infrequent email users.

> to get them to talk to each other.

This leaves the question of whether you want to have internet bandwidth
soaked up by spam, just so you can have more ham input for your filter
learning system?

[...]
> they certainly are no single solution.

I've been tempted to set up an IPv6-only MTA, and see how much spam it
gets when 'disconnected from the public Internet' ;-)

> resources than many access-based filters written either in PERL, or

Heh. You don't know where I work. We make a hardware product that does
content analysis at OC48 speeds, with guaranteed throughput. Content
inspection is a processing intensive operation, compared to a binary
search for a token in an access list, no matter what the implementation
language (or hardware) for the filter is.

> admins want to run what works for them.  This doesn't belong in a

In contrast, people build firewalls built into routers, not as seperate
devices. Additionally, there are some things you can do by being inline
in the process, that you can't do as an adjunct afterwards.

							David

---

From: Andreas Saurwein <[email-masked]>
Date: Tue, 28 Oct 2003 16:10:53 -0200

>receive (if there are other options for eliminating some quantity of

May I direct you to http://www.theregister.co.uk/content/54/25715.html ??
Pay special attention to the small paragraph which reads:

"According to CCC's data, up to 60 per cent of laptop claims received are 
for accidental damange. The rest are for theft. From this it extrapolates - 
on the assumption that there are five million laptops in circulation in the 
UK (a bit high, maybe?) that 100,000 are damaged and c.67,000 are stolen 
every year. Most thefts apparently occur in offices and schools (12 per 
cent each)."

I would not call approx. 67.000 stolen laptops by year only in the UK a 
"quite rare" event.
May you receive just one spam mail from each of these notebooks :)

Cheers
Andreas

---

From: "Tom Thomson" <[email-masked]>
Date: Tue, 28 Oct 2003 18:37:46 -0000

> May you receive just one spam mail from each of these notebooks :)

But it doesn't really matter, because any sane anti-spam system using email
source authentication will not try to do sender authentication at the
receiver.  The sending MUA authentication needs to be done by the sending
MTA, the receiver then verifies that the sending MTA is allowed to transmit
mail from that address.

It will make some types of roaming a bit harder.  But, unlike smtp-auth and
other such end-to-end authentication systems, it will be usable for
assigning responsibility for spam.

Tom

---

From: David Maxwell <[email-masked]>
Date: Tue, 28 Oct 2003 14:48:24 -0500

> I would not call approx. 67.000 stolen laptops by year only in the UK a 

67,000 / 5,000,000 = 1.34%   Also, as a percentage of email users as a
whole, it's even tinier.

You don't estimate how many of those laptops aren't immediately
formatted and resold. Sure, if identity theft was key to spammers
propagating spam, maybe more laptops would be stolen - that of course
completely ignores my comments about ISPs limiting the number of
messages sent to some order of magnitide within the normal behaviour of
a user.

I suppose 1.34% isn't 'quite rare' it's only 'pretty uncommon', hmm?

							David

---

From: Brad Knowles <[email-masked]>
Date: Wed, 29 Oct 2003 12:41:38 +0100

>  something outside of that consent definition.

	Depends on how you define "mail from these four people", and how 
you prove beyond a shadow of a doubt that the message did actually 
come from them.  What about messages forwarded by them from other 
people?  Does the message have to originate uniquely from one of 
these four people, or are they allowed to forward messages that they 
themselves did not write?

>  and b) can't ever be perfect.

	I submit that authentication methods can also be CPU intensive, 
and usually also have a small percentage of failures, both false 
negatives and false positives.

---

From: Brad Knowles <[email-masked]>
Date: Wed, 29 Oct 2003 13:03:59 +0100

>  determine from your posts is that you are opposed principle to any

	I am opposed to useless modifications that will serve no purpose 
in the long-run, and which will cause all e-mail users to give up 
essentially all liberties in the process.

	If you have actually useful proposals to make, I'm all ears.

>  high in the form of stone tablets, and that we're all going to h*ll

	You do an excellent job of intentionally mis-interpreting and 
mis-construing the views and words of others, and then casting them 
as the one true Satan against which everyone *MUST* obviously be 
opposed.

	You should get a job in the Bush administration.

>  help address the spam problem, and not result in eternal damnation?

	There is no "one true absolute solution", not with the methods 
and tools available to us today.

	You can throw out the entire protocol and start over, with the 
attempt to create such an absolute solution, but that is not the 
purpose of this group.

>  Is there ANY possibility that an anti-spam proposal will someday

	Actually, some of the tools already available to us today do the 
best job that I think could be reasonably expected.

	I think the real key is to lay out very clearly what kinds of 
things should not be done, what the consequences are if you do them 
anyway, and make recommendations as to how people can configure their 
software so as to take information from the broadest array of 
reasonable input sources and therefore make the most informed 
decision it could reasonably make.

---

From: "Alan DeKok" <[email-masked]>
Date: Wed, 29 Oct 2003 12:14:53 -0500

Brad Knowles <[email-masked]> wrote:
> >  help address the spam problem, ....
> 

  "help address the spam problem" != "one true absolute solution"

> attempt to create such an absolute solution, but that is not the 

  I never said that, either.

> mis-construing the views and words of others,

  Hmm...

  Alan DeKok.

---

From: Brad Knowles <[email-masked]>
Date: Wed, 29 Oct 2003 14:37:50 +0100

>  200+ emails in one day be enough to shut him down?

	Trojan a million machines, have them watch the outbound traffic 
for a few days, then configure them to send an average of about 1x 
that amount as spam.  That should be a low enough value to by-pass 
any such per-user rate limits.

	Moreover, this puts us into a similar situation where the RIAA is 
suing 12-year old girls and grandmothers, because we have 
mis-identified their apparent unusual activity, or because they did 
things (or their system did things) and they were not aware of the 
consequences.

>  change would be significantly beneficial to the network as a whole.

	At best, it would make it marginally more difficult to use open 
proxies, but spammers would just trojan a larger number to arrive at 
the same results.  Or, they'd get smarter and pool their resources 
and get into outsourcing arrangements.  Or any or all of the above.

	And that would only be at the ISPs where they implement such 
per-user rate limiting controls, which would be expensive and 
difficult to implement.  You could do a whitelist for ISPs that do 
implement such per-user rate limiting, but you'd have to make sure 
that it's not implemented via the DNS (otherwise you get into all 
sorts of nasty cache poisoning issues, such as have been previously 
discussed).

>  soaked up by spam, just so you can have more ham input for your filter

	Bandwidth-wise, spam has never shown up on any measures of 
network concerns, and never will.  A few browsings of porno websites 
will generate far more traffic than you'd ever receive in a single 
day.

	The cost to ISPs has nothing to do with bandwidth utilization. 
It has to do with transactional costs (handling X messages per 
second/minute/hour/day), storage costs, CPU load caused by trying to 
reduce spam, etc....  The cost to individuals does sometime have a 
bandwidth cost, but again that rapidly disappears in the noise when 
compared to web traffic, etc....