Subject: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)

From: Shelby Moore <[email-masked]>
Date: Sat, 06 Sep 2003 20:22:03 +0800

Request for opinions on whether to creating a working group or publish the following idea as an internet draft?

Spam is big problem that is getting worse.  BrightMail.com (which claims to process 10% of world's email) claims that the percentage of spam out of all email has grown from 16% in Jan. 2002 to 50% in Aug. 2003.

A fundamental unsolved problem of doing any thing about spam, is there is currently no unambiguous definition of spam as an enforceable internet standard.  This has been architectually impossible to define because the receiver is the subjective determinant of which bulk email is solicited and which is spam (UBE).

ISPs, Hosts, legislators, judiciaries, and even anti-spam software, have a fundamental problem in that definition of spam as UBE is currently architectually unenforceble due the fact that subjective determination of "unsolicited" current happens after the email has been delivered to the receiver.

My idea is to create an internet draft, RFC, and hopefully internet standard, that would define a simple architectual paradigm for legitimate bulk email that unambiguously separates it from spam (UBE).

Simply define that legitimate bulk distribution of email should be done by mechanism of each bulk distributor providing a public POP3 (and IMAP) account or server, rather than sending the email directly.

In the case of a public distribution (e.g. most direct email and mailing lists), a POP3 (and IMAP) account of user "anonymous" with password "none" would suffice.  In the case of private dissemination (private mailing lists), a POP3 (and IMAP) server with individual accounts could be provided.

The elegance of this paradigm is that users then control the opt-in/opt-out database, by configuring their email client to POP email from only the bulk POP accounts they wish to subscribe to.

The effort to support this paradigm is minimal because it uses existing email paradigm.  Legitimate bulk senders have to change from a broadcast ("push") metaphor (e.g. Majordomo) to a "pull" metaphor simply by depositing their outgoing email in the public POP account they create.  Receivers simply follow instructions to POP bulk email they want, instead of the equally complex task of subscribing to bulk email.

This accomplishes several goals:

1. Any bulk email is then spam (receiver has not opted in) and can be dealt with by ISPs, Hosts, legislators, judiciaries, and anti-spam software.
2. Receivers now have uniform control over opt-in/opt-out policy without a global authority
3. Legitimate bulk senders can be insured that they or their email won't be misclassified as spam
4. Those who send UBE can no longer claim they are legitimate or that receiver has opted-in (ambiguity removed) and can be dealt with by ISPs, Hosts, legislators, judiciaries, and anti-spam software.
5. With a "pull" paradigm, the load (resource usage) on the public internet, sender, and receiver is reduced, because I venture that a majority of bulk email sent would not be pulled.

I think this paradigm would empower Hosts, ISPs, legislatures, and judiciaries to do more about spam (incoming) and spammers (outgoing), because their hands would not longer be bound by ambiquity.  I realize that some vested interests, such as direct emailers or those invested in push based mailing lists, might resist.  However, I think the benefits outweigh the limited costs to migrate.  Some direct emailers might resist because some may prefer being able to cloak spam under the guise of "solicited".  Legitimate bulk emailers stand to gain a lot by separating themselves from the noise of UBE.

Shelby Moore
http://AntiViotic.com

---

From: [email-masked]
Date: Sat, 06 Sep 2003 20:17:56 -0400

On Sat, 06 Sep 2003 20:22:03 +0800, Shelby Moore <[email-masked]>  said:

> Simply define that legitimate bulk distribution of email should be done by
> mechanism of each bulk distributor providing a public POP3 (and IMAP) account
> or server, rather than sending the email directly.

This is broken in two distinct ways:

1) I as a mail user now have to go check 150 POP servers several times a day
for all the various lists I'm on - many of the lists are low-volume, but I'd have
to go CHECK every day just in case something DID get posted.

> In the case of a public distribution (e.g. most direct email and mailing
> lists), a POP3 (and IMAP) account of user "anonymous" with password "none"
> would suffice.  In the case of private dissemination (private mailing lists), a
> POP3 (and IMAP) server with individual accounts could be provided.

Nope.  even for a public list you get to keep a separate POP3 account for each
subscriber - if one person has checked for postings yesterday, but another hasn't
since last Tuesday, you can't feed the right list to each person.

So that brings us to:

2) I as the administrator of a site that hosts 6,000 mailing lists just got the
additional aggrivation of providing POP3 service for 700,000 e-mail addresses
(yes, we've got that many).  This includes "My password doesn't work" support
and things like that.  Gee thanks.

> The elegance of this paradigm is that users then control the opt-in/opt-out
> database, by configuring their email client to POP email from only the bulk POP
> accounts they wish to subscribe to.

Have you actually *TRIED* to use more than 100 POP accounts under any current
mail software?

> 1. Any bulk email is then spam (receiver has not opted in) and can be dealt
> with by ISPs, Hosts, legislators, judiciaries, and anti-spam software.

So I drop a note to 50 friends inviting them to a barbecue, and I end up in the slammer.

> 2. Receivers now have uniform control over opt-in/opt-out policy without a global authority

This actually means "We've pushed the headache to the recipients".

> 3. Legitimate bulk senders can be insured that they or their email won't be

So.. you ready to have every single eBay or Amazon customer check their POP account
there every day just in case there's important mail for them?

> 4. Those who send UBE can no longer claim they are legitimate or that
> receiver has opted-in (ambiguity removed) and can be dealt with by ISPs, Hosts,
> legislators, judiciaries, and anti-spam software.

Well.. maybe.  But..

> 5. With a "pull" paradigm, the load (resource usage) on the public internet,
> sender, and receiver is reduced, because I venture that a majority of bulk emai
> l sent would not be pulled.

So let's see.. Currently, if your bank sells your e-mail address to another company,
you get spammed.  So instead, you'll have it so that you check your bank's POP
server in case there's important mail about your mortgage.  Seems like the obvious
scheme is for the bank to charge the other company to put stuff in your POP mailbox.

So you still get spammed...

> their hands would not longer be bound by ambiquity.  I realize that some vested
> interests, such as direct emailers or those invested in push based mailing
> lists, might resist. 

It's ironic that you're proposing this on a push-based mailing list provided by
an organization that is probably not in a position to provide POP accounts for
the 30,000 or so recipients of the the list.

Baby with the bathwater, Shelby... Baby with the bathwater.

---

From: Shelby Moore <[email-masked]>
Date: Sun, 07 Sep 2003 09:58:47 +0800

Thanks for the feedback and giving me a chance to clarify some issues.

>This is broken in two distinct ways:

Disagree.  Read on.

>to go CHECK every day just in case something DID get posted.

If it became an RFC or internet standard, and it became widely adopted, then it is reasonable to assume that email clients would add features to handle this.  It is quite a low bandwidth operation (probably less than 1K bytes) to poll a POP server for email.  I assume it would become popular because as shown with further logic in my responses below, the idea provides strong benefits to all (except spammers).

However, there is one key technological hurdle I did miss in my haste, there would need to be some mechanism so that the same user doesn't keep downloading the same messages over and over again.  This would either require a special modification to the POP server and require each user to login with a unique user name. Or better, users' email clients can be made smarter because there is a UIDL command in both POP3 and IMAP4.  This unique identifier can be used by the email client to only download messages which are new to that user.  One would assume that POP servers could also remove messages older than say 1 month or so (configurable by the administrator).

And as a side benefit, there would be no way for someone to subscribe me to a list without my permission, as can be done by sniffing an authentication email for Majordomo.  And no way for someone to subscribe me to a list that has no public instructions for subscribing or unsubscribing (i.e. spam in guise of business email).

>since last Tuesday, you can't feed the right list to each person.

False.  You are correct that I missed this issue in my initial post.  However, it need be only one POP account (one storage of emails) with flags for each user.  In other words, the storage requirement need no increase drastically with number of subscribed users.  The flags can either be stored at the POP server and then give each user a unique login id, or more realistically just let email clients manage their own flags using UIDL.

>and things like that.  Gee thanks.

No only 6000 POP accounts.  See above how email clients can handle the detection of new messages using UIDL.  And you only need one anonymous login and no password (just configure the POP server to accept any login and password).

Now instead of sending 700,000 emails for each email sent to all your 6000 of you lists, you only send 6000 emails or less.  Now instead of managing bounces, keeping your IP off of blacklists, hassling with subscribing and unsubscribing the users, then all you have to do is publish the domain names of your 6,000 POP servers on a web page.  The flow of noise is probably greatly reduced.

>Have you actually *TRIED* to use more than 100 POP accounts under any current

I will respond with similarly rhetorical question.  Did you try to use Netscape 2 on most current web pages?  Why make any application RFC if there can be no progress in applications?

>So I drop a note to 50 friends inviting them to a barbecue, and I end up in 

50, even 5000, is not statistically bulk on internet scale.  Is it not possible (or likely) to write laws without exclusions?  Do you think Hosts, ISPs, and anti-spam software would not account for this statistical phenomenon?

>This actually means "We've pushed the headache to the recipients".

How so?  In my mind, I find it to be no more of a headache than subscribing and unsubscribing to a mailing list.  And certainly a lot less of a headache than trying to opt-out of a list that won't let you opt-out.

I think recipients already have a big headache, it is called "spam".  And it is getting worse.  It is predicted that very soon 90% of all email sent will be spam.  When that happens, the internet email system will start to fail as a viable communication medium.  Then your 6000 mailing lists won't work any more any way.

>there every day just in case there's important mail for them?

Is it any different than checking their existing POP account(s) and reviewing InBox each day??

And as an advantage, they can (if they want) with a more finely grain comb choose which email to check and when.

>So you still get spammed...

No.  Because you can chose to not check it and/or you at least know who is spamming you and can hold them responsible directly.  Thus your bank would stop doing it, because they make $ by not losing your business.

>the 30,000 or so recipients of the the list.

No.  As I said above, they would only need to provide one POP account for this mailing list.

>Baby with the bathwater, Shelby... Baby with the bathwater.

No baby, bathwater only.  Check the logic above.  Just the facts, Valdis...Just the facts.

---

From: [email-masked]
Date: Sat, 06 Sep 2003 23:07:44 -0400

On Sun, 07 Sep 2003 09:58:47 +0800, Shelby Moore said:

> If it became an RFC or internet standard, and it became widely adopted, then 
>it is reasonable to assume that email clients would add features to handle this
> .  It is quite a low bandwidth operation (probably less than 1K bytes) to poll 
> a POP server for email

It's not the bandwidth - it's the fact that there are these annoying things called
"timeouts".  For *each* server that isn't reachable, you get to wait a minute or
so - suddenly those 150 checks are taking quite some time.  And the queuing theory
of 150 sites with intermittent connectivity doing a push to your POP server is
different than what you see when you try to do 150 pulls....

But of course, if you actually *tried* this, you'd understand it...

> However, there is one key technological hurdle I did miss in my haste, there 
> would need to be some mechanism so that the same user doesn't keep downloading 
> the same messages over and over again.  This would either require a special modification
> to the POP server and require each user to login with a unique user name.

Hey.. what did I tell you?  Everybody needs a login of their own...

> And as a side benefit, there would be no way for someone to subscribe me to a
> list without my permission, as can be done by sniffing an authentication email
> for Majordomo. 

If your confirmation mail is being sniffed, you have *BIGGER* issues.

And if you have bigger issues, I suggest using the *proper* tools for the job.
See RFCs 2362-2364 and 3156.  If your issues are bigger than that, e-mail
subscriptions are the least of your problems.

> False.  You are correct that I missed this issue in my initial post.
> However, it need be only one POP account (one storage of emails) with flags for
> each user.  In other words, the storage requirement need no increase
> drastically with number of subscribed users.

Hmm... store each mail as an object with links for each recipient.  A truly novel
idea, our homegrown mail system implemented it back in 1992.

>                                             The flags can either be stored at the POP server 
> and then give each user a unique login id,

Hey.. there's that unique login again.

> No only 6000 POP accounts.  See above how email clients can handle the
> detection of new messages using UIDL.  And you only need one anonymous login
> and no password (just configure the POP server to accept any login and
> password).

And I tell *MY* UIDL from Keith Moore's UIDL from Vernon Schreyer's UIDL how?

> I will respond with similarly rhetorical question.  Did you try to use
> Netscape 2 on most current web pages?  Why make any application RFC if there
> can be no progress in applications?

There's a difference - I'm not proposing a new scheme of doing things that
involves a change to 500 million users.  So I submit to you that if this idea
is too hard to use with the current version of Outlook, it is a *non starter*
as a practical matter.

> 50, even 5000, is not statistically bulk on internet scale.  Is it not
> possible (or likely) to write laws without exclusions?  Do you think Hosts,
> ISPs, and anti-spam software would not account for this statistical phenomenon?

Only problem is that many spammers are *already* only dropping 40-50 copies
of a note at a site at a time, specifically to work around that - then the rest of the
spamming recipients at your site get a different version with a different From: line
and a different source IP address.

I submit to you that if you didn't realize this was happening, you may not be qualified
to be suggesting proposals to counter it....

(Hint - if spammers weren't doing this, it would be trivial to block them, and we'd
not be HAVING this discussion, right? ;)

> No.  As I said above, they would only need to provide one POP account for this mailing list.

And as I pointed out, you'll need to create 30,000, because one account doesn't
allow you to keep track of who has already seen what messages.  And no, you're
*NOT* allowed to just say "everybody can fetch all the UIDLs and we'll just tag
them with the subscriber ID" - go read and *UNDERSTAND* section 6.2 of RFC2298
in order to understand why.

You might also want to go re-read the ASRG mailing list archives, your proposal
(and variants thereof) has been kicked down the beach like a dead whale
multiple times already.

---

From: [email-masked]
Date: Sat, 06 Sep 2003 23:42:51 -0400

> in order to understand why.

Another reason why you need unique userid/logins for each subscriber - so that
you can prevent forging a UIDL for somebody else to keep them from reading the
message.  Being able to do this (and if you have a shared userid, it's almost
impossible to prevent) would make the Bernstein/Bush flamefest regarding list
censorship look trivial in comparison...

Oh... there's also this thing called "webmail".  Lots of people use them so they can
get mail no matter where they are.  Lots of these people will be fundamentally stuck
under your proposal, because every time they used a kiosk they'd have to enter
in all 20-30-40 POP servers they had mailing lists on.  Surprisingly enough,
hotmail.com and yahoo.com have a lot of subscribers, you might want to factor
that into your plan....

---

From: Shelby Moore <[email-masked]>
Date: Sun, 07 Sep 2003 11:38:19 +0800

>different than what you see when you try to do 150 pulls....

First, you are using a baseline of 150 in your example, which is attuned perhaps to your power use given you stated that you manage 6,000 mailing lists.  I doubt the average user would subscribe to more than 10 on a regular basis.

Second, there is a way of dealing with timeouts on the client side, it is called multithreading.  Multithreading (for those readers who might not know, if any) is a fundamental programming concept that all internet clients (and all computer programs and operating systems) depend on.  Thus the culmulative delay is not O(n), it is more like Max(n).

Thirdly, with your queuing point, you are making the mistake of assuming that the majority of email received is *legitimate* bulk email.  I assert this is far from the case for the majority of people.  Any way, for the power cases like yours, what is the big deal with setting your email client to run for a few minutes each day, while you go browse some research at google.com or fill your coffee cup?  Is your personal paradigm of immediacy for legitimate bulk email a good priority for the internet as a whole, when the very nature of email itself is being doomed by spam?

>But of course, if you actually *tried* this, you'd understand it...

Please stop the personalized attacks and stick to the facts.

>Hey.. what did I tell you?  Everybody needs a login of their own...

Er...you conveniently ignored the "or" that followed the "either":

-----I wrote (and you ignored)-----
Or better, users' email clients can be made smarter because there is a UIDL command in both POP3 and IMAP4.  This unique identifier can be used by the email client to only download messages which are new to that user.  One would assume that POP servers could also remove messages older than say 1 month or so (configurable by the administrator).

>If your confirmation mail is being sniffed, you have *BIGGER* issues.

Again you conveniently ignored the overall point which followed.

-----I wrote (and you ignored)-----
And no way for someone to subscribe me to a list that has no public instructions for subscribing or unsubscribing (i.e. spam in guise of business email).

That seems to be a BIG problem that many people have these days, probably even yourself.

>Hey.. there's that unique login again.

Er...you conveniently ignored the "or" that followed the "either":

-----I wrote (and you ignored)-----
, or more realistically just let email clients manage their own flags using UIDL.

>And I tell *MY* UIDL from Keith Moore's UIDL from Vernon Schreyer's UIDL how?

Yes your email client can keep track of which UIDL's it has already downloaded.  Keith's email client can keep track of his.  Vernon his.

Ironically you mention Vernon (is that a hint?), because if I remember correctly he runs the DCC and recently had a similar unexplained overtly negative reaction recently when I tried to contact him regarding some improvements to DCC.  I suppose it is possible that your negative responses here may be explained by similar vested interest with him.  Maybe, but in fairness I won't immediately assume the worst ethics of you.

BTW, in case you think my anti-spam business depends on this proposal, that would be false assumption.  Actually one of the key advantages of my anti-spam algorithm (the automatic white listing of legitimate bulk email) would be rendered unnecessary if this proposal was put into effect.

>involves a change to 500 million users.

They all had to upgrade their browsers.

>  So I submit to you that if this idea

If Microsoft likes the proposal, it won't matter what you or I think :)

There are many RFCs that are not implemented by Outlook.  That doesn't stop people from trying innovate and then let the chips fall where they will.

I think that current version of Outlook could be used as is for bulk of users (who only have a few legitimate bulk email needs), except that old messages would get continually redownloaded.  But fortunately Outlook supports plugins, so it should be fairly trivial to add the UIDL tracking to Outlook, even if Microsoft refused to.

>and a different source IP address.

So you are indeed heavily involved with the DCC!  Yes I know that is a weakness in the way the DCC measures bulk, but that is irrelevant to the point above.

>to be suggesting proposals to counter it....

If I did not realize it was happening, then I would not have created http://AntiViotic.com .   I'd be using the DCC instead.  I also wouldn't have approached Vernon about improving the DCC (only to be turned away in nasty tone).

>not be HAVING this discussion, right? ;)

As I said above, I actually have a business incentive not to make this proposal.  But I felt it needed to be made.  My conscience and desire to do important work is stronger than my desire for profit.

I wish you would remove the personalized attacks in your posts and stick to a discussion of facts.

>in order to understand why.

You did not understand the concept.  The per user UIDL tracking is stored in the email client, not in the POP server.

>(and variants thereof) has been kicked down the beach like a dead whale

Well apparently not enough, since you are claiming to be fully knowledgeable about those posts, and you are not able to provide good refuting logic to my proposal yet.

If you would like to refer me to some specific threads (with links please), then I will be happy to read them and adjust my comments here or even withdraw the proposal if I find it is warranted.

Shelby Moore
http://AntiViotic.com

---

From: Shelby Moore <[email-masked]>
Date: Sun, 07 Sep 2003 11:51:19 +0800

>censorship look trivial in comparison...

Again I will repeat:

You apparently did not understand the concept.  The per user UIDL tracking is stored in the email client, not in the POP server.

>get mail no matter where they are.

You can get mail no matter where you are with a POP account also.

Most people use webmail either because it is free, or because it is a convenient interface on their POP account.

>  Lots of these people will be 
>fundamentally stuck
>in all 20-30-40 POP servers they had mailing lists on.

Again cars today have rubber tires and titanium rims, not wooden wheels.

Applications can improve.  It actually happens most of the time.

Hotmail and Yahoo already support and encourage the pulling of email from POP accounts. You don't have to enter the POP accounts every time, just once.

>  Surprisingly enough,

How many more silly personal attacks?

---

From: [email-masked]
Date: Sun, 07 Sep 2003 00:21:38 -0400

On Sun, 07 Sep 2003 11:51:19 +0800, Shelby Moore said:

> Hotmail and Yahoo already support and encourage the pulling of email from POP
> accounts. You don't have to enter the POP accounts every time, just once.

Hmm.. so Hotmail is willing to maintain the list of 40 or 50 places it has to POP your
mail from for you?  Or are you re-entering it every time you use a new kiosk someplace?
(Remember that the major business case for webmail is that you have a *VERY*
small amount of state carried with you (a userid/password pair - specifically so you
can check mail with a minimum of local resources).

---

From: [email-masked]
Date: Sun, 07 Sep 2003 00:29:43 -0400

> Please stop the personalized attacks and stick to the facts.

I didn't make a personalized attack.  I merely pointed out that in all probability,
you haven't actually *tried* doing what you suggest, because it's not anywhere
near as usable as you might think.

... and then...

> So you are indeed heavily involved with the DCC!  Yes I know that is a
> weakness in the way the DCC measures bulk, but that is irrelevant to the point
> above.

If I'm involved with the DCC in any way shape or form other than the fact I've
seen Vernon write about it on IETF lists, it's such a covert involvment that
even I am unaware of it.

---

From: John C Klensin <[email-masked]>
Date: Sun, 07 Sep 2003 00:42:17 -0400

--On Saturday, September 06, 2003 8:22 PM +0800 Shelby Moore 
<[email-masked]> wrote:

> Spam is big problem that is getting worse.  BrightMail.com (which claims
> to process 10% of world's email) claims that the percentage of spam out
> of all email has grown from 16% in Jan. 2002 to 50% in Aug. 2003.
>
> A fundamental unsolved problem of doing any thing about spam, is there is
> currently no unambiguous definition of spam as an enforceable internet
> standard.  This has been architectually impossible to define because the
> receiver is the subjective determinant of which bulk email is solicited
> and which is spam (UBE).
>...

Shelby,

Valdis has identified some of the technical issues associated with using 
POP3 in this way.   Let me step back and look at your proposal from another 
angle.

>From the standpoint of the bulk mailer/ poster of material, there is no 
advantage (and some disadvantages) of doing that posting so that interested 
users can "pull" it relative to just posting the material on a web page 
somewhere.   Functionally, what you have proposed seems, to me, to be 
roughly equivalent to:

	* distributors of bulk materials are required to post them on
	selected web sites.
	
	* non-bulk materials may continue to go out via 	conventional email.

Nice plan.  The problem is that spammers won't play and efforts to coerce 
them into playing will largely fail due to international issues, lack of 
adequate incentives, etc.... exactly the same problem we have today with 
state laws prohibiting spam.

More generally, you have just defined an "opt in" model that assumes that 
anyone who has not explicited opted to receive particular messages will be 
able to get them (or be sent them) only be some overt action on the 
would-be recipient's part.  We know from experience that such a model won't 
work without significant legal pressure and enforcement -- if you don't 
believe me, sample any reasonable quantity of spam for messages that claim, 
quite strongly, that, if you hadn't opted in, you wouldn't be receiving it.

Sorry, but no cigar.

     john

---

From: Shelby Moore <[email-masked]>
Date: Sun, 07 Sep 2003 12:15:45 +0800

At 11:53 PM 9/6/2003 -0400, you wrote:
>Actually, the point is that there was no way, even within usenet, to 
>prevent pollution of individual groups with innappropriate spam or off 
>topic messages.  Many groups have fallen into disuse for this reason.

Afaics, that is irrelevant, because even a mailing list can be polluted with spam.  Your point is against the nature of a public list or group, but my proposal is not designed to fix that problem.  My proposal is designed to fix the problem of receivers being forced to receive bulk email from any sender.  My proposal forces email addresses to not be public to legitimate bulk email.  And then all other bulk senders can be dealt with more effectively.  Then of course this does solve the problem for mailing lists also in the end, because then all bulk email gets killed (by a combination of legislative, judicial, ISP, Host, and software features).  The fundamental problem with doing any thing about spam, is there is no way to measure it, because the subjective measurement of "unsolicited" is not architectually defined.

The point is that if I choose to receive email from list or group (irregardless of whether that list or group has effective policies to prevent incoming spam posts), then it would be helpful to *architectually* differentiate that which I subjectively decide is legitimate bulk email senders from that which I decide is UBE (spam) senders.

Once you define the architecture, then all kinds of important features can be built on top of it to deal more effectively with spam ranging from legislative, judicial, ISP, Host, and software.

>The other point to understand is that the paradigms (usenet vs email) are 
>and have been different for very good reasons.  And if one assumes that 
>both paradigms are desired, then it is necessary to address how one 
>enforces the separation.  No one to date has figured out even a set of 
>definitions to address the needs, much less mechanisms.

I am unclear how you think this applies to my proposal?

Shelby Moore
http://AntiViotic.com

---

From: Shelby Moore <[email-masked]>
Date: Sun, 07 Sep 2003 12:23:19 +0800

>mail from for you?

I can not predict what Hotmail will do.  But given they have an incentive to get you to come their web site as often as possible, and given it is the same bandwidth cost for them, with the potential of making their service more valuable, then my guess is they would do it.

Interestingly note that Hotmail makes you pay to POP *FROM* hotmail, but no charge to POP from other accounts *TO* Hotmail.  Does that give you any hint about their business model??

>  Or are you re-entering it every time you use a new kiosk 
>someplace?

If they are going to offer the feature, then no reason for them not to persist the configuration.  They would have no incentive to offer the feature and make it impossible to use efficiently.

>can check mail with a minimum of local resources).

Now you are supporting my case above.

---

From: Shelby Moore <[email-masked]>
Date: Sun, 07 Sep 2003 12:30:00 +0800

> I merely pointed out that in all 
>probability,

How could you know if does not yet exist?

We can discuss facts and theories of how it would work.  But no one can actually test how it does work, because it does not exist yet.

The comparison of setting POP accounts in an email client and popping the email, versus subscribing to that many bulk email lists and popping the email, is not useful because no email client has yet been designed to optimize popping from many anonymous POP accounts.

>seen Vernon write about it on IETF lists, it's such a covert involvment that

Ok cool.  Let's move on.

Shelby Moore
http://AntiViotic.com

---

From: shogunx <[email-masked]>
Date: Sat, 6 Sep 2003 01:45:58 -0400 (EDT)

Hello Shelby,

I run a few mail servers, and have built many more. I personally would
have no desire for my mail to be handled by POP3, passed in cleartext
across the public internet, when I simply log into
my machine securely (locally or remotely) and type "mail" to access my
email.  Further, I am not interested in having my mail sit on someone elses
server.  Don't get me wrong Shelby, I HAVE a POP3 server, that I built
myself, for the use of friends and affiliates.  Nor do I have the desire
to collect 2-3000 mail boxes in my lifetime, for all of the business
relationships I may make during this period.

IF you want to get rid of spam, try communism, wherein there is no need
for commercial messages of any type.  Seems easier than a complete rewrite
and redeployment of SMTP, which is what would be necessary inthis
instance.

Scott

---

From: Shelby Moore <[email-masked]>
Date: Sun, 07 Sep 2003 12:56:04 +0800

>POP3 in this way.

I have refuted all of Valdis's technical points so far.

>   Let me step back and look at your proposal from another 
>angle.

Yes I think that is productive to discuss the end game (outside of technical issues).

>	* distributors of bulk materials are required to post them on

It is an acceptable analogy, except I disagree that bulk posters of material favor web post over POP post.  I think they favor what ever their receiver favors, by the law of supply and demand and economics.  If receivers find it more convenient to get a copy in their InBox, then that will be accomodated (just as with mailing lists and archives now).

But in terms of the analogy, I will follow along...

>	* non-bulk materials may continue to go out via 	conventional email.

Agreed.

>Nice plan.  The problem is that spammers won't play

Agreed.

> and efforts to coerce 

Here is where I *strongly* disagree.

Let me start with a story.  The genesis for this proposal came from the fact that our outgoing business email (not bulk but single emails sending a password to a customer, etc) is being blacklisted by SPEWS (etc) because Rackspace (our Host) allowed some other customers of theirs to send spam on the same C class IP range as ours.  SPEWS then blacklists the entire C class.  Well SPEWS is uncorrectable lately because they've been under DoS attacks (from spammers presumably), thus caches of blacklists are used and nothing can be done except for us to change our email IP address.  So in discussing this with the AUP manager and her boss at Rackspace, it became clear that Rackspace would never be able to guarantee the quality of a C class range of IPs, because "Rackspace can not determine what is legitimate bulk email and what is spam and thus can not terminate new customers until a very heavy proof of spamming has already occured, by which time the damage to C class has already been done".

So the lesson learned was that if Rackspace could automatically detect high quantities of bulk email in real-time, then with my proposed architectual change, Rackspace could in real-time shut off the spammer.

Okay so that is one example of how the architectual paradigm changes the rules and allows more effective actions against spammers.

Now take for example legislative combined with ISP.  For right now, spammers are avoiding open relays and many foreign IPs because of blacklists, so they get a dialup account and send from there.  Well if there was a law requiring USA ISPs to detect and shut these off in real-time, then spammers would need to revert back to open relays and foreign IPs which are effectively dealt with using blacklists.  Then blacklists would not have to be so draconian with IP C ranges in countries with strict enforcement, which would make the blacklists more effective and useable.

Then take anti-spam software like the DCC, BrightMail, or even our AntiViotic.com.  If we know all bulk is bad, the game gets simpler because no whitelist needed.  Since whitelists are data that is forgeable by spammer, this closes another hole.  No to mention that whitelists make current anti-spam less useable and realistic on wide scale.

I could go on... but I hope you begin to see how everything to fight spam depends circularly on the ability to architectually define it.

If you can't measure it, you can't do any thing about it.  That is a fundamental datum of science.

>would-be recipient's part.

That is the definition of opt-in.

>  We know from experience that such a model won't 

What you are saying IMO, is that you can't force bulk emailers or spammers to use opt-in.  That has been because you can't measure the spam (UBE) from the legitimate.

It is a chicken and egg problem.  Once you have the egg (the architectural metric), then reasonable to make the chicken.  So comparing to before you had the egg, is not necessarily illustrative.

>Sorry, but no cigar.

I am smoking it (figuratively) right now :)

Thanks for getting to the crux of the matter and allowing me to make it clear.

Shelby Moore
http://AntiViotic.com

---

From: Shelby Moore <[email-masked]>
Date: Sun, 07 Sep 2003 13:07:10 +0800

>email.

There is nothing in my proposal that requires your private email to deviate from your current more secure methods of SMTP delivery.

By definition, any thing that is bulk is no longer private, because you can't control what another person will do with the same information.

>  Further, I am not interested in having my mail sit on someone elses
>server.

Your email would not.  Only bulk email which is shared with many other people.

>  Don't get me wrong Shelby, I HAVE a POP3 server, that I built

It is a wrong assumption to equate business email with bulk email.

>for commercial messages of any type.

It is a wrong assumption to equate commercial email with bulk email.

>  Seems easier than a complete rewrite

Don't get me wrong, but respectfully, this has *NOTHING* to do with SMTP.  SMTP is not involved and not changed.

Shelby Moore
http://AntiViotic.com

---

From: [email-masked]
Date: Sun, 07 Sep 2003 01:42:11 -0400

On Sun, 07 Sep 2003 12:23:19 +0800, Shelby Moore said:

> Interestingly note that Hotmail makes you pay to POP *FROM* hotmail, but no
> charge to POP from other accounts *TO* Hotmail.  Does that give you any hint
> about their business model??

Yes.  It's *NOT* a business model where they want to be polling a dozen servers
on a regular basis for each of their customers for mail that may or may not be
there, and for the average mailing list, probably is not there at any given
poll.  They want eyeballs, and the last thing they want to do is expend more
effort than needed to get eyeballs. Sure - they can even optimize the 'POP the
list' check by only doing it once for all the subscribers - but they're still
hitting each server for each list on a several-times a day basis.  And under
the current scheme, they can just *catch* one SMTP transaction with all the
RCPT TO's piggybacked *when there's actual mail*.  So they'd have to work a lot
harder under your scheme.

And let's *THINK* for a moment here - what is your proposal *REALLY* going to
change?  We already have many estimates that 50% or so of all e-mail is spam.
Let's take that as a given, and let's make the rash assumption that the rest is
25% mailing list traffic and 25% person-to-person.

So what you want to do is take the 25% of the list traffic that works just fine
on the current infrastructure, and is usually quite easily whitelistable via a
number of different methods - and move it to something totally different.
And what you're left with is a 2-1 mix of spam and personal mail that you
yourself admit things like the DCC and spam filters are unable to perfectly
distinguish.  To quote Douglas Adams: "Many reasons for this unhappiness were
proposed, most of which involved the movement of small green pieces of paper,
which was odd given that on the whole, it wasn't the small green pieces of
paper that were unhappy..."

Having exiled the mailing list traffic,  we would then be able to work on
separating the spam from non-spam - but as you already noted yourself, we don't
know how to do that yet.  And getting rid of the mailing list traffic doesn't
in fact gain us anything at all, since everybody who filters list traffic into
separate folders for each list knows that isn't the problem - it's the
unfiltered stuff that's left in the inbox.

I'll note in passing that the two highest SpamAssassin scores I've ever seen
were both on legitimate postings to mailing lists -  both were humor pieces
about spam....

Quite frankly, given that at least half the spam I get is already in obvious
violation of at least one law (pick one - securities fraud, advance-fee scams,
wire fraud, bogus pharmeceuticals, or hijacking a proxy to send the mail), I
severely doubt that anything the IETF does in regards to standards won't make a
difference. The spammers often don't even bother following RFC822 - why should
they follow your scheme?

The *only* two ways to get rid of spam both involve making it non-profitable.

The first is lowering the generated income.  Given that recently, somebody
hacked the site of a "fertilizer for your body part" scam, and found a list of
6,000 people who had paid $50 a bottle, I have to sadly conclude that Korbluth
and Barnum were both correct, there's one born every minute and the rate is
increasing.  So there's no joy to be found there.

The second is raising the cost to the spammer.  Personally, I like the idea of
taking up a collection among the ISPs and other providers, and hiring some good
ethnic muscle (there's competition in the field, a number of experienced and
ruthless groups are available).  I'm sure the spam problem would change
drastically if the spammer was seriously having to balance the mentioned $300K
for bogus enhancement pills against having their kneecaps broken by one group
or worse by one of the other groups...

Pity that will never work though.  At least not officially (although one infamous
New Zealander apparently retired recently...)

---

From: [email-masked]
Date: Sun, 07 Sep 2003 01:43:37 -0400

> It is a wrong assumption to equate commercial email with bulk email.

Which is why you're trying to rewrite how bulk email is done in order to deal
with *one segment* of commercial e-mail.  Now I understand fully.

---

From: grenville armitage <[email-masked]>
Date: Sun, 07 Sep 2003 14:31:51 +1000

[..]
> >(and variants thereof) has been kicked down the beach like a dead whale
	[..]
> If you would like to refer me to some specific threads (with links please), then
> I will be happy to read them and adjust my comments here or even withdraw the
> proposal if I find it is warranted.

Valdis gave you the initial pointer, why should he do your literature review
(prior art?) research as well?

And (re-ordering just a little) I find it a little wierd to see:

	[..]
> Please stop the personalized attacks and stick to the facts.

And then a few lines down you opine (of Valdis's motives)...

> I suppose it is possible that your negative responses here may
> be explained by similar vested interest with him.  Maybe, but in
> fairness I won't immediately assume the worst ethics of you.

Hmmm.

gja

---

From: Shelby Moore <[email-masked]>
Date: Sun, 07 Sep 2003 14:02:30 +0800

IMO, this (whether Hotmail will implement a specific feature) is a fairly irrelevant (an 80 out of 80/20 rule) fork of the debate relative to the main point of the proposal, so let's try to wrap this fork up with one or two go rounds max okay.

>poll.

Not any more than they want to be POPing any email at all.  Nobody wants to do any work they do not have to do.  But if there is advantage for them, or a profit to be made, they do it.  If they do not, someone else will, then they lose marketshare.

They used to not POP email at all (do you remember or did you know that?).  Then they discovered they were missing a big market of eyeballs.

>  They want eyeballs, and the last thing they want to do is expend more
>effort than needed to get eyeballs.

No disrespect intended, but that sentence is illogical.  You want something and you don't want to do something that gives you what you want.

You mean I guess that they would not agree to add effort just to retain existing eyeballs.  Again I disagree.  I think they will do what ever they have to in order to retain market share, as long as the cost doesn't kill more profit than it retains.

> Sure - they can even optimize the 'POP the

POPing once (one list mailing) versus processing one email with zillion RCPT TOs (one list mailing) is not a very big cost difference.  One might be slightly less than the other and we really can't say which one, but it is irrelevant because the difference is insignificant.

Actually it is more likely that when they POP they will get several messages at once, so less cost than catch several SMTP emails.  

Also they know a priori the correlation of receivers to POP, which can be optimized with time, versus having to build a new mapping table in real-time every time they process an SMTP with RCPT TO.

>25% mailing list traffic and 25% person-to-person.

It be more interesting to know what the real stats are on the other 50%, because I doubt that 25% is legitimate bulk email.  It seems that you live in a different (mailing list centric) world than I and most "normal" people live in.  I join mailing lists for a short time to get something done, then I leave asap.  Most of the people I know and the many thousands of customers I come into contact with, seem to not even know how to use a mailing list.

With 500 million people on the internet, I would venture that 80% don't even know what a mailing list is.  They may use Yahoo Personals, and not even realize it is a mailing list.  Since the email is being directly deposited into the Yahoo account, they have no clue.

Any way, let's follow your line of debate...

>on the current infrastructure,

No it doesn't work fine.  My gf complained that she couldn't find her Yahoo Personals email amongst the 500 spams she gets per day, of course that makes me happy but that is besides the point :)

> and is usually quite easily whitelistable via a
>number of different methods -

Whitelisting can be subverted by spammers:

http://www.cnn.com/2003/TECH/internet/09/01/spam.chainletter/index.html

"...Herrick, however, admits that the practice could be a good way to bypass e-mail filters which block messages from senders who are not known to the recipient. Spammers could use chain letters to discover the addresses of people with whom you frequently communicate. Spam purporting to be from someone in your address book would sneak by filters. 
"If I were a spammer, I'd be working very hard to perfect this technique," he said..."

> and move it to something totally different.
>distinguish.

The whole point of the change is to enable elimination of the spam which can not currently be done.

See my response to John C Klensin, regarding "chicken and egg" and the example benefits to attacking spam:

http://www1.ietf.org/mail-archive/ietf/Current/msg22050.html

>know how to do that yet.

Yes we probably do.  Just because the DCC can not measure bulk email reliably doesn't mean Hosts, ISPs, and other software can not.  BrightMail already is (just signup for an Earthlink account and try really hard to get some spam), and I will also be probably be demonstrating something soon.

>  And getting rid of the mailing list traffic doesn't

You are missing the point, which is until you can say that all bulk email is spam, then you can't target spam.  How could ISPs, Hosts, legislators, and judges know the difference between legitimate email and spam?  Again see the targeting benefits:

http://www1.ietf.org/mail-archive/ietf/Current/msg22050.html

Worse as it stands now, mailing list traffic can often get misidentified as spam, unless it is a well established list.

>were both on legitimate postings to mailing lists -  both were humor pieces

I've already written publicly in 2002 that the Bayesian and any content filtering methods cause more harm than they solve.

>difference. The spammers often don't even bother following RFC822 - why should

Again you are missing the whole point.

It is has nothing to do with what spammers will or will not do.  It has to do with what Hosts, ISPs, etc are currently prevented from doing.  Since they can not determine what is spam, they can not enforce any law.  The practicalities of blocking email based on a wide range of hard to prove laws is none.  There would be too much liability for the enforcer if they do not successfully win the criminal case.  Whereas if you have a simple, clear cut metric as in my proposal, then ISPs, Hosts, etc can take action and will take action because spam is one of their major costs.

>The first is lowering the generated income.

I agreed.  I have written a thesis on this entitled, "Fragile (yes I think so!) Economics of Spam"

>  Given that recently, somebody

I've read the theories and you realize that the spammer's margins are very thin.  It won't take too much to topple the boat.  The problem is that the architecture is not adequate to increase their costs significantly yet.  That is why I made this proposal.

>The second is raising the cost to the spammer.

Agreed 100%.   See above.

>  Personally, I like the idea of

That is not an effective deterent as evident by the drug war and crime in general.  You actually have to make it more expensive to send than to recieve.  That is what my proposal is all about.

>New Zealander apparently retired recently...)

:-)

---

From: shogunx <[email-masked]>
Date: Sat, 6 Sep 2003 03:05:46 -0400 (EDT)

> There is nothing in my proposal that requires your private email to deviate from your current more secure methods of SMTP delivery.

Shelby, by doing things this way i get no bulk email.  Noone in their
right mind would target a server with 15 users for spamming... this
happens to mail providers with millions of subscribers.

> By definition, any thing that is bulk is no longer private, because you can't control what another person will do with the same information.
Why don't we try to eliminate it all in one fell swoop... bulk email, junk
postal mail, telemarketing calls, the whole lot of it.  There is an
annoying company called the social security something or other who for
years sent me certified letter after certified letter saying that I should
have one of their numbers, and please contact them.

> Your email would not.  Only bulk email which is shared with many other people.
Good, because I don't want any bulk mail on my server.  You can collect
all you want for me on some random server though.  Not interested in
wasting either the cpu cycles or disk space it would take to deal with it.

> It is a wrong assumption to equate business email with bulk email.

-------------------------------------------------------------------------------
#!/bin/sh

rant(){
echo (
I am of the opinion that I never want to be sold anything in my life.  If
I want it, I can find it.

These people deploying spammers as marketing agents are no different than,
or possibly are, the people who deployed call center sweatshops as
marketing tools, and bulk mailers eating thousands of trees proclaiming
what great deals they have on cars this week.  The truth is, if someone
wants to spam bad enough, they will.  We could rewrite smtp 5 times, and
it would be engineered, reverse-engineered, reengineered, implemented,
modified, patched, and kludged by thousands of people for various reasons
each time.  And every time one of those reasons would be spam.  As long as
we live in a profit-driven society anyway.  Its too bad we can't just
hold hands and sing cumbyah, but it happens that way sometimes.  This has
been rehashed time and time again on this list, unfortunately, which is far
from the venue for this discussion.

Who volunteers his mail server for the spam-discussion-redirect list that
will effectivly be the /dev/null for spam related topics that arrive at
the ietf?  For those of you unfamiliar with UNIX or any of its
derivatives, /dev/null is a storage device for important internal system
processes.  What say you IETF?  Can we engineer a solution to this nasty
problem?
) | /bin/mail -s spam [email-masked]
sleep 12
rant
}

rant

-------------------------------------------------------------------------------

> It is a wrong assumption to equate commercial email with bulk email.
rant

> Don't get me wrong, but respectfully, this has *NOTHING* to do with SMTP.  SMTP is not involved and not changed.
Therin lies the flaw in your plan, as smtp must continually change in a
distributed fashion in order to effectively reduce the amount of
egregiously time-wasting email that flows through its veins.

> http://AntiViotic.com

sleekfreak pirate broadcast
world tour 2002-3
live from the pirate hideout
http://sleekfreak.ath.cx:81/

---

From: Vernon Schryver <[email-masked]>
Date: Sun, 7 Sep 2003 00:43:06 -0600 (MDT)

> From: Shelby Moore <[email-masked]>

>>And I tell *MY* UIDL from Keith Moore's UIDL from Vernon Schreyer's UIDL how?

How did I get involved in this?

15 years ago I had a boss that finally taught me to never use real
names in examples or scenarios even when I was sure I was being
nice.  It's better to invent names, and to take care that they're
far from anything real.  You must assume that someone will take
you literally in bad ways you'll never predict.

> ...
> Ironically you mention Vernon (is that a hint?), because if I remember
> correctly he runs the DCC and recently had a similar unexplained
> overtly negative reaction recently when I tried to contact him regarding
> some improvements to DCC.  I suppose it is possible that your negative
> responses here may be explained by similar vested interest with him.
> Maybe, but in fairness I won't immediately assume the worst ethics of you.

"Worst ethics"?
Mr. Moore contacted to ask me to sign a non-disclosure agreement so
that I might learn how to make the DCC "effective" based on his new
"intellectual property."  He was also interested in buying copies of
DCC data.  My responses started cool and eventually became as "overtly"
negative as I could without calling him names.  I told him that he
had disclosed nothing of his intellectual property to me, that there
is no plausible chance we might ever do any business, that I wished
he would stop sending me mail, that I did not (and do not) want to
hear about his intellectual property, and that his lawyers should be
aware that my document retention policy for email is "forever in a
bank vault" and that I've been known to take formal notes during
telephone calls.  Perhaps Mr. Moore's harping on his "intellectual
property" and an NDA sent me into a raving paranoid break, but many
of us know real life stories that started similarly and turned out
poorly from some points of view.

He sent a few more messages after my request that he stop, but eventually
quit.  If this triggers more, I'll file them with the others, not in
file 13 but in that bank vault.  Media's cheap today.

I'm writting this only to urge caution.  Some people are innocent and
well meaning and only appear otherwise to us paranoids (split personality
and delusions of royal or editorial grandeur as well).  Other people
really are dangerous.  A few years dealing with spammers or with
patents and intellectual property experts should make anyone spooky.
I realize the most dangerous people don't seem dangerous.  Maybe that
proves Mr. Moore's virtue, or not.

> BTW, in case you think my anti-spam business depends on this proposal,
> that would be false assumption.  Actually one of the key advantages
> of my anti-spam algorithm (the automatic white listing of legitimate
> bulk email) would be rendered unnecessary if this proposal was put
> into effect.

I hope I don't need to go read about that automatic white listing
and so forth.  I find Mr. Moore's technical writing as inpenetrable
and painful to read as my own.  For now I'll assume that the IETF
archives provide sufficient protection.

> ...
> So you are indeed heavily involved with the DCC!  Yes I know that
> is a weakness in the way the DCC measures bulk, but that is irrelevant
> to the point above.

Please notice my resolute failure to ask about that weakness.  I've
no clue what Mr. Moor's intellecutal property is, except that I suspect
it is similar to some ideas related to counting mail from IP addresses
mentioned by others in public...and conveniently archived by Google.

> If I did not realize it was happening, then I would not have created
> http://AntiViotic.com .   I'd be using the DCC instead.  I also wouldn't
> have approached Vernon about improving the DCC (only to be turned away
> in nasty tone).

That's me, the ol' evil, nasty, non-team playing, selfish, negative,
unfair, anti-comerz, kook himself.

Another thing that Mr. Moore convinced me during our exchanges is that
his notions about the nature of the DCC works differ significantly
from mine.  There's nothing wrong with that, since without Mr. Moore's
intellectual property, the DCC is not "effective" or whatever.

I hope the fact that every time I turn around I bump into another
person demanding attention for the Ultimate Wonderful Final Perfect
Solution to The Spam Problem is a good sign instead of a perverted
maturation of the anti-spam industry.

Vernon Schryver    [email-masked]

---

From: shogunx <[email-masked]>
Date: Sat, 6 Sep 2003 03:12:20 -0400 (EDT)

> or worse by one of the other groups...

What, who needs a kneecap job done?  Contact me off-list;)  We have very
reasonable rates....

wait.  sorry.  that was spam.

---

From: Shelby Moore <[email-masked]>
Date: Sun, 07 Sep 2003 14:21:52 +0800

>with *one segment* of commercial e-mail.  Now I understand fully.

No I am trying to eliminate (drastically reduce) spam (UBE).  And, I am trying to give receivers the ability to opt-in and opt-out of all legitimate bulk email under their own power.

In my mind, that is more important than existing legitimate bulk email paradigm.  I can understand you and other's vested interest in  existing legitimate bulk email paradigm, but I don't agree it is more important.

When spam is 99% of all email you recieve, you may begin to care also.  It won't be too long from now...

Fortunately what this discussion is proving to me so far, is that architectual change will be resisted fiercely by vested groups that control the design of the internet.  This means that my new (soon to be patent pending) algorithm for http://AntiViotic.com will have a near monopoly in the market.  So for me and users of my service it will be great.  But for the rest of users, spammers will be forced to actually increase the amount of email they send in the self-feeding model of my algorithm.  Unfortunately, the rest of the email universe will be imploded by an accelerating rate of spam.  Just imagine what spammers will do when confronted with an insubvertable algorithm that can detect bulk.  I came here to see if there was a better way, but I guess we will have to do it the hard way (and very profitable for me).  This is all conjecture (vaporware) at this point...but very well researched conjecture.

I'll be back here in this list later (probably a year from now) when your needs have changed to a more dire state regarding email.

---

From: Shelby Moore <[email-masked]>
Date: Sun, 07 Sep 2003 14:37:42 +0800

At 01:54 AM 9/7/2003 -0400, you wrote:
>The evidence indicates that the senders will use whatever is more likely to 
>result in the receiver seeing the message.  This is different from seeing 
>it where the receiver would like to see it.

I get your point and it is a reasonable one that must be taken seriously.

However, I think that at some point soon, users will have no choice but to either stop using email (because it doesn't work efficiently any more) or look for a solution to spam.

At that point, I think they will demand change (already seeing early warning signs of that now).

Also you are assuming that legitimate bulk email is very important to most receivers wherein I think for most spam is a bigger issue.

Lastly, I don't think my proposal has to be any more inconvenient than existing sub/unsubscription procedures, which require trading several emails, versus inputting one POP domain in an email client.

I understand the concept of diminishing utility in economics, so I take your point seriously.  And I will ponder it more.

>The point of the usenet reference analogy is that if placing things where 
>people have to go look was sufficient, folks would not have started using 
>spam email.

But there is a key difference.  Users started preferring email without the agreement that it would have spam rate increase from 8% to 50% in 2 years.  When the noise gets too high, you turn off the TV and do something more productive (remember antenae reception?).  If you have a blackout every 5 minutes, you giveup trying to do anything with appliances.  Etc..

>  They would have used other techniques.

And they will if spam continues to increase.  They already are.  And those techniques may have much worse effects than a correct architectual definition of spam.

For example, legitimate mailing list traffic is getting falsely classified as spam.  Maybe not for you, perhaps because you are expert, but I assert your experience is not likely shared equally by the 500 million...

>  Many (most) of the 
>spammers are simply interested in any technique that will get the message 
>in front of the recipient.  Hence, they will use the path that is checked 
>most often (the personal mailbox).

Yes my proposal depends on that fact.  Once you have the legitimate email separated from the spam architecturally, then you can effectively increase the cost of spamming to the point it is a non-economically viable activity.

Remember the economics of spam is precisely what makes it so noisy.  If spam ever got like 5% response rate, then it would not be such a big problem as it is.  The fact that is averages something like < 0.005% response rate is what makes it both a big noise problem and also makes it very fragile cost wise.  The typical individual spammer does not make $300,000 a day.  More like less than $100,000 a year.  And the institutionalized spammers (the economy-of-scale bundlers) would make nice big fish for FTC.  The more you can concentrate, the better.

Shelby Moore
http://AntiViotic.com

---

From: Shelby Moore <[email-masked]>
Date: Sun, 07 Sep 2003 15:17:13 +0800

>egregiously time-wasting email that flows through its veins.

No you are counting the chickens before the eggs.  If you have no more eggs, then you won't get any more chickens.

The misuse of SMTP has to be attacked at the root cause, which is my proposal (go study all my posts for full logic), rather than morphing SMTP in a never ending game of "cat and mouse".

---

From: Shelby Moore <[email-masked]>
Date: Sun, 07 Sep 2003 15:19:05 +0800

There are some false characterizations that I can not leave unrefuted in public...

>Mr. Moore contacted to ask me to sign a non-disclosure agreement

No I contacted you to ask if I could have access to the public IP data that is shared between all DCC servers.  I told you I would be using it to validate or invalidate an anti-spam algorithm I have devised.  And I told you that I did not want to reveal the algorithm to you.  You were skeptical so I said that the only way I could reveal more was under NDA.  Then you flipped out and even refused my further requests for clientID and password, which is (or was) advertised from your site to be a publicly available resource.  Go figure!

> so
>"intellectual property."

I said that my algorithm could complement/improve the DCC by providing automatic whitelisting.  You interpreted that statement incorrectly.

I also said that the DCC must already be effective else you would not have so much market share.

>  He was also interested in buying copies of
>DCC data.

I said that if you weren't willing to give me the data you share publicly for free, then I would be willing to contribute to the DCC to pay for any effort to gather the data I needed.  You apparently misinterpreted my considerate and respectful demeanor as some attempt 
to buy you off.

>  My responses started cool and eventually became as "overtly"
>negative as I could without calling him names.

Agreed.

>  I told him that he
>had disclosed nothing of his intellectual property to me,

Agreed.  You signed no NDA, so why would I.  I did not need to.  I only needed to get some statistical data for testing an algorithm.  I offered that it might help in the overall fight against spam.  You could have been more respectful and cordial in declining my request.  Given I was only asking for data that is already public, it still makes no sense to me.  But no matter, I have moved on and devised a better plan to get data.

> that there
>is no plausible chance we might ever do any business,

I never wanted to do business and I told you that.  I only wanted to get some public data for testing.

> that I wished
>telephone calls.

Which I obliged after telling you I hate attorneys and I don't have any.

If I had an attorney, I wouldn't be allowed to post in the IETF mailing list.  Just go read the IP release for posting to the mailing list.

>  Perhaps Mr. Moore's harping on his "intellectual
>property" and an NDA sent me into a raving paranoid break,

You were harping on how skeptical you were and I was just trying to convince you I had something important while also saying I could not reveal IP.  It would seem to me that if you have public data and you are devoting your life to fighting spam, then you would do as much useful work as you possibly can with that data.  But any way, that is your problem, not mine.  I've moved on.

> but many

How many of us get killed in automobile accidents (top 3 killer), yet we still drive.  And IP is not deadly.  No logical comparison statistically.

>quit.

I know the technique.  Make a bunch of false statements then demand no replies in hopes you have a case in future.  I had an obligation to set record straight for my email file before quitting.  I told you exactly what I was doing.

>  If this triggers more, I'll file them with the others, not in

Please do.  More chance to make sure the record is clear.  Too bad you had to do it in public which compels me to respond in public.  But I only need this one post.  You can post any response and there is no need for me to respond again.  All crucial false claims by you in public have been refuted.

>really are dangerous.

So are cars.

>  A few years dealing with spammers or with

Yes I can imagine that.  <joke>I will like see "them" come to where I am currently (remember I told you I was temporarily out of the USA), a place where the only law is my relative who will knock them off for mere $100 or a pig and few bottles of beer. :)

>proves Mr. Moore's virtue, or not.

Thanks.  You will eventually realize I am harmless except to spammers.

>archives provide sufficient protection.

Please do not.  My businesses are irrelevant to this proposal.  I made a proposal to try to delineate legitimate bulk email from UBE.  That is all.

>mentioned by others in public...and conveniently archived by Google.

No comment.

>maturation of the anti-spam industry.

Well either you are lot smarter (experienced) than me, or that seems like the comment of an embedded skeptic.  They say if we do the same thing for too long, our brain or anything we utilize become unable to do or view other things with an open mind.

In closing, let by gones be gone.  I have no more ill feelings towards what transpired between us.  I will merely prove you wrong. :)

Damn it is a small world.  I had no idea I would run into you here.

Shelby Moore
http://AntiViotic.com

---

From: [email-masked]
Date: Sun, 07 Sep 2003 03:45:09 -0400

On Sun, 07 Sep 2003 14:02:30 +0800, Shelby Moore said:

> POPing once (one list mailing) versus processing one email with zillion RCPT
> TOs (one list mailing) is not a very big cost difference.  One might be
> slightly less than the other and we really can't say which one, but it is
> irrelevant because the difference is insignificant.

When you compare  a successful SMTP to a successful POP, yes.

> Actually it is more likely that when they POP they will get several messages 
> at once, so less cost than catch several SMTP emails.  

Actually, the most likely is "POP once an hour for a once-a-week posting, and
you've totally blown 167 pointless transactions".  And in fact, unless you're able
to make the POP check frequency less than the posting frequency, you'll lose.

> "If I were a spammer, I'd be working very hard to perfect this technique," he said..."

And your proposal does nothing to stop this, since it won't look like bulk mail,
it will look like personal mail.   From people you know and trust, and everything.

> Yes we probably do.  Just because the DCC can not measure bulk email reliably
>  doesn't mean Hosts, ISPs, and other software can not.  BrightMail already is (
> just signup for an Earthlink account and try really hard to get some spam), and
> I will also be probably be demonstrating something soon.

So why do we need to move off mailing lists, when the problem is solved?

> It is has nothing to do with what spammers will or will not do.  It has to do

Actually, it has everything to do with what they will or wont do.

> with what Hosts, ISPs, etc are currently prevented from doing.  Since they can
> not determine what is spam, they can not enforce any law.

And separating out mailing list traffic doesn't change matters, really.

You're left with a lot of non-bulk high-volume business e-mail (yes, you WANT
this to get through, otherwise Amazon doesn't have a way to tell you easily
about a problem with your order, or similar), a lot of person-to-person mail,
and a lot of spam pretending to be one or the other of the above.  The only
thing you've cut out is spam to mailing lists - and if you can solve the OTHER
two flavors above, then this third is a non-issue anyhow.

Given that so much spam is already breaking some law, why do you think "the ISPs
could enforce a NEW law" would make any difference?

From another note:

> Yes my proposal depends on that fact.  Once you have the legitimate email
> separated from the spam architecturally, then you can effectively increase the
> cost of spamming to the point it is a non-economically viable activity.

This would be wonderful, except what you're separating isn't legitimate/spam,
it's mailing list/non mailing list.  The two are orthogonal concepts.

*plonk* Somebody wake me up if Shelby starts addressing the actual problem,
rather than an orthogonal non-problem....

---

From: Shelby Moore <[email-masked]>
Date: Sun, 07 Sep 2003 16:11:57 +0800

>And in fact, unless 
>you're able

That is quite an easy optimization to do.

Study Probability Theory and Statistics (a higher level math class).

>it will look like personal mail.

*plonk* wrong again.  That about roughly 20 *plonks* in a row for you.  You have not made one factually correct statement yet in this thread.

The bulkness as measured by ISPs and Hosts (those who can enforce the law in real-time) is done by looking at statistics on IPs and packets.  It has nothing to do with the mutable fields of email, such as headers or content.

If you don't believe that email can be reliably statistically profiled by the source ISP, then go Google before you respond.

>So why do we need to move off mailing lists, when the problem is solved?

Because I wrote "bulk".  I did not write "legitimate bulk".  Current algorithms have no way to differentiate the two.  Remember that was the whole point of this proposal.  *plonk* you don't even remember what the subject of this thread is nor do you read my posts carefully.

>Actually, it has everything to do with what they will or wont do.

But that is the reaction, not the action.  The action (which is first step needed) is done by enforcers...

>And separating out mailing list traffic doesn't change matters, really.

*plonk* again.

I did not write "separate mailing list traffic".  I proposed to "differentiate legitimate bulk email from Spam (UBE)" (see the Subject line above).

And if you can separate the legitimate bulk traffic from the UBE traffic, then the existing enforcers (BrightMail, DCC, Hosts, ISPs, etc) can take action against all remaining bulk which is then all UBE.

>about a problem with your order, or similar), a lot of person-to-person mail,

Agreed all non-bulk.

>and a lot of spam pretending to be one or the other of the above.

Which is bulk and can now be attacked by enforcers because the legitimate bulk emailer will not be harmed in process, because it is separated.

>  The only
>thing you've cut out is spam to mailing lists

*plonk* again

We've moved the legitimate bulk email to "pull".  The spam (UBE) gets attacked and reduced by existing enforcers that know how to detect bulk.  And then non-bulk is untouched.  That is the ideal solution.  Nothing orthogonal at all.  You think it is orthogonal because you are not visualizing how the architecture change actually works to reduce (or eliminate) spam.

>could enforce a NEW law" would make any difference?

Example.  If I have a customer I am Hosting who is sending bulk email (which I as Host or ISP can measure statistically), but that customer claims the email is legitimate.  There is nothing I can do in real-time, automated way to refute his claim.  The cost for me to attack the spam is greater than the cost I will save.  Therefor I punt and say "spam vs. legit bulk email is ambiguous".

Whereas if I have a clear rule that all bulk is spam (UBE), then I can make more profit by setting up automated, real-time, statistical blocking.  Now I drive the cost of spamming higher, eventually to a point that it is no longer profitable.  Not just one enforcer or one type of enforcer, but all the examples I gave in my posts take in unison.

If you want to understand any likely action, always study the economics.

>it's mailing list/non mailing list.  The two are orthogonal concepts.

*plonk* again

I do not how you got so confused.  Read above.

>rather than an orthogonal non-problem....

Is that enough *plonk*s to wake you up?

At this point, the discussion is getting redundant and boring.  So I think I will be signing off soon...perhaps not to answers any more posts... so I think my posts already suffice to make my case, irregardless of more *plonks* that will inevitably follow this one...

Shelby Moore
http://AntiViotic.com

---

From: grenville armitage <[email-masked]>
Date: Sun, 07 Sep 2003 21:54:17 +1000

Shelby Moore wrote:
	[..]
> I join mailing lists for a short time to get something
> done, then I leave asap.

This appears to be the root of our problem.
At least there's an upside.

gja

---

From: Keith Moore <[email-masked]>
Date: Sun, 7 Sep 2003 10:20:32 -0400

On Sun, 07 Sep 2003 12:56:04 +0800
Shelby Moore <[email-masked]> wrote:

> 
> What you are saying IMO, is that you can't force bulk emailers or spammers
> to use opt-in. 

Let's be even clearer.  What's being claimed is that you can't force bulk
emailers to send their email via "pull" technology (whether this means
providing their own POP servers, IMAP servers, NNTP servers, web servers,
whatever) while everyone else can still use "push".  And the question isn't
really whether bulk mail can be statistically distinguished from non-bulk
mail (since that's really just a matter of whether you can get people to
adopt a definition of "bulk" in terms of externally visible traffic
properties) - the question is whether you can enforce that rule.

IMHO - most recipients don't want to get their mail that way (and many of
the deployed user agents don't support it), most senders don't want the
increased burden of providing POP/IMAP/NNTP/web servers and the necessary
customer support, and there are enough ISPs that derive significant revenue
from selling bandwidth to spammers that it would be extremely difficult to get
them all to enforce this.  In short: nobody has sufficient incentive to adopt
this.

Of course there's nothing wrong with defining another way to distribute bulk
mail that people can use if they wish.  If it works well, some people will
use it.  The stretch is to insist that everybody do it this way.

---

From: shogunx <[email-masked]>
Date: Sat, 6 Sep 2003 16:07:52 -0400 (EDT)

> I'll be back here in this list later (probably a year from now) when your needs have changed to a more dire state regarding email.
Thank you for playing.

---

From: Dean Anderson <[email-masked]>
Date: Sun, 7 Sep 2003 15:45:23 -0400 (EDT)

Information theory says that such things are impossible.  One can not
construct a spam-free protocol because this is the same problem as
constructing a system free of covert channels, which information theory
says is impossible.  It is not simply hard. It is impossible, like
perpetual motion.

After I first posted this on IETF a while back, someone suggested that
covert channels require cooperation, and that spam therefore isn't a
covert channel.  I have recently discussed the issue at length with some
PhD's and other very smart people from one of my old companies, and we
determined after a great deal of discussion that covert channels (and
similar concepts like side channels and information leakage, and other
terms used in information theory literature)  don't require cooperation as
a necessary condition.  The term "Covert Channel" is mostly used in
liturature analyzing operating systems. However, the general concept is
referred to in non-OS liturature as "Side Channels" or "Information
Leakage", or "Sneaky Channels". The concepts are essentially the same.

Indeed spam can be modeled as a communication channel that is multiplexed
with email, which is itself multiplexed with TCP, which is multiplexed
with IP, which may be multiplexed over a variety of physical media.  It
was also realized (towards the end) that spam is in fact a cooperative
covert channel, since the reader cooperates when the receive the spam and
open it for reading.  Only _after_ reading the spam message are some
people offended, but this is of no consequence to the application of
information theory because the communication has already taken place. All
they can do, having now detected the abuse, is try to suppress further
abuse that is similar.

But this is a simpler way to think about it:  Spammers can continue to
claim they are legitimate emailers, because they _ARE_ legitimate, so far
as we know before they send email. And even so far as we know _before_
someone _READS_ their email.  Only after reading their email, and perhaps
only after some investigation, can we know for sure that the sender and
message is conducting abuse or in violation of their AUP.

Some might ask about blocked abuse. Subsequent abuse is blocked because it
is similar to previous abuse (by IP address with blacklist), or by content
with content filters.  We can anticipate some likely abusive content, but
cannot identify all abusive content in advance.

However, I looked at your proposal, and it appears that you are trying to
create a "pull" mechanism rather than a "push" mechanism for message
delivery . This paradigm has already been implemented as "Usenet News".
It is not immune to spam, though it distributes spam and other broadcast
messages much more efficiently than typical email systems.  However,
bandwidth consumed by spam email (indeed bandwidth consumed by _ALL_ email
combined), is still minor, so we are really not in need of a more
efficient means of distributing spam, or email for that matter.

		--Dean

On Sat, 6 Sep 2003, Shelby Moore wrote:

> A fundamental unsolved problem of doing any thing about spam, is there
> is currently no unambiguous definition of spam as an enforceable
> internet standard.  This has been architectually impossible to define
> because the receiver is the subjective determinant of which bulk email
> is solicited and which is spam (UBE).
>
> ISPs, Hosts, legislators, judiciaries, and even anti-spam software, have
> a fundamental problem in that definition of spam as UBE is currently
> architectually unenforceble due the fact that subjective determination
> of "unsolicited" current happens after the email has been delivered to
> the receiver.
>
> My idea is to create an internet draft, RFC, and hopefully internet
> standard, that would define a simple architectual paradigm for
> legitimate bulk email that unambiguously separates it from spam (UBE).
>
> Simply define that legitimate bulk distribution of email should be done
> by mechanism of each bulk distributor providing a public POP3 (and IMAP)
> account or server, rather than sending the email directly.
>
> lists), a POP3 (and IMAP) account of user "anonymous" with password
> "none" would suffice.  In the case of private dissemination (private
> mailing lists), a POP3 (and IMAP) server with individual accounts could
> be provided.
>
> The elegance of this paradigm is that users then control the
> opt-in/opt-out database, by configuring their email client to POP email
> from only the bulk POP accounts they wish to subscribe to.
>
> The effort to support this paradigm is minimal because it uses existing
> email paradigm.  Legitimate bulk senders have to change from a broadcast
> ("push") metaphor (e.g. Majordomo) to a "pull" metaphor simply by
> depositing their outgoing email in the public POP account they create.
> Receivers simply follow instructions to POP bulk email they want,
> instead of the equally complex task of subscribing to bulk email.
>
> 1. Any bulk email is then spam (receiver has not opted in) and can be
> dealt with by ISPs, Hosts, legislators, judiciaries, and anti-spam
> software.

> receiver has opted-in (ambiguity removed) and can be dealt with by ISPs,
> Hosts, legislators, judiciaries, and anti-spam software.

> 5. With a "pull" paradigm, the load (resource usage) on the public
> internet, sender, and receiver is reduced, because I venture that a
> majority of bulk email sent would not be pulled.
>
> I think this paradigm would empower Hosts, ISPs, legislatures, and
> judiciaries to do more about spam (incoming) and spammers (outgoing),
> because their hands would not longer be bound by ambiquity.  I realize
> that some vested interests, such as direct emailers or those invested in
> push based mailing lists, might resist.  However, I think the benefits
> outweigh the limited costs to migrate.  Some direct emailers might
> resist because some may prefer being able to cloak spam under the guise
> of "solicited".  Legitimate bulk emailers stand to gain a lot by
> separating themselves from the noise of UBE.
>
> Shelby Moore
> http://AntiViotic.com
>
>
>

---

From: Shelby Moore <[email-masked]>
Date: Mon, 08 Sep 2003 04:24:30 +0800

>At 11:51 AM 9/7/2003 +0800, you wrote:
>shelby, that's actually not true. If you have an enterprise email service 
>that requires access to a VPN and the internet service you access it with 
>(e.g hotel room ethernet) has a bad firewall configuration, you may never 
>get to the mail. I speak with personal experience - the hotel I am in right 
>now has screwed up its firewall. I ended up having to find an 802.11 hotspot 
>to get to my email.

I understand but that was not my point.  My point is that you can put a web-based interface on top of your POP account to access it any where.  You still have a POP account which you are accessing any where if that is what you want.  The web-based interface is just another form of an email client.

The point is that you don't need to use a web-based email without an underlying POP account in order to access email from any where.  There are even places where HTTP web-based interface won't work (e.g. cell phone) and so you need to use a different form of email client to access.  Still you can have an underlying POP account that mail is being drawn from.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Sun, 7 Sep 2003 22:44:53 +0200

On zondag, sep 7, 2003, at 21:45 Europe/Amsterdam, Dean Anderson wrote:

> says is impossible.

Nobody cares. Making a roof 100.000000% impervious to water molecules 
may be impossible, but that doesn't mean we have to resign to getting 
wet every time it rains.

> It is not simply hard. It is impossible, like perpetual motion.

So when exactly was the earth supposed to stop moving?

> covert channel.

Where does this covert channel stuff come from anyway?

> message is conducting abuse or in violation of their AUP.

This goes for each individual message, but the spammer's achilles heel 
is that they need to send out incredible amounts of email in order to 
fulfill their objectives, whichever those are. Detecting bulk mail is 
doable, and it shouldn't be too hard to come up with something to 
differentiate legitimate bulk emailing from spam. For instance, we can 
reverse the burden of proof here and only allow know bulk emailers.

> delivery . This paradigm has already been implemented as "Usenet News".

My point exactly except that usenet doesn't have to be significantly 
more "pull" than email (where you need your client to check for new 
mail as well).

> messages much more efficiently than typical email systems.

Ouch! :-)

Fixable with authentication.

---

From: "vinton g. cerf" <[email-masked]>
Date: Sun, 07 Sep 2003 17:07:11 -0400

>I understand but that was not my point.  My point is that you can put a web-based interface on top of your POP account to access it any where.  You still have a POP account which you are accessing a…

that's different - what you said was as quoted above. I agree that if you design the web server properly, you can use a web interface, but you run the risk that with this design, you may never be able to pull the email later, POPStyle, into your computer. Although it is theoretically possible, using POP (rather than IMAP) to leave the mail on the server until you pull it again with POP, many servers appear to clear out the mail after POPing it. I think John Klensin made that observation in an earlier exchange.

>The point is that you don't need to use a web-based email without an underlying POP account in order to access email from any where.  There are even places where HTTP web-based interface won't work…

see above.
v

Vint Cerf
SVP Technology Strategy
MCI
22001 Loudoun County Parkway, F2-4115
Ashburn, VA 20147
703 886 1690 (v806 1690)
703 886 0047 fax
[email-masked]
www.mci.com/cerfsup

---

From: Shelby Moore <[email-masked]>
Date: Mon, 08 Sep 2003 04:53:10 +0800

>>I understand but that was not my point.  My point is that you can put a 
>web-based interface on top of your POP account to access it any where.  You 
>still have a POP account which you are accessing any where if that is what 
>you want.  The web-based interface is just another form of an email client.
>
>that's different - what you said was as quoted above.

>>>>You can get mail no matter where you are with a POP account also.

You are writing about different issues of which email client is ideal for each scenario and what the feature set needs to be.  My statement is still true.  Programmers can massage (program) the email client as needed, then my statement is never false.  I assert there are already email clients to access POP from most any where, even if they aren't well known.

> I agree that if you 
>design the web server properly, you can use a web interface, but you run the 
>risk that with this design, you may never be able to pull the email later, 
>POPStyle, into your computer. Although it is theoretically possible, using 
>POP (rather than IMAP) to leave the mail on the server until you pull it 
>again with POP, many servers appear to clear out the mail after POPing it. I 
>think John Klensin made that observation in an earlier exchange.
>
>
>>The point is that you don't need to use a web-based email without an 
>underlying POP account in order to access email from any where.  There are 
>even places where HTTP web-based interface won't work (e.g. cell phone) and 
>so you need to use a different form of email client to access.  Still you 
>can have an underlying POP account that mail is being drawn from.
>
>see above.

---

From: John C Klensin <[email-masked]>
Date: Sun, 07 Sep 2003 17:38:15 -0400

--On Sunday, September 07, 2003 17:07 -0400 "vinton g. cerf"
<[email-masked]> wrote:

>> I understand but that was not my point.  My point is that you
>> can put a web-based interface on top of your POP account to
>> access it any where.  You still have a POP account which you
>> are accessing any where if that is what you want.  The
>> web-based interface is just another form of an email client.
> 
> that's different - what you said was as quoted above. I agree
> that if you design the web server properly, you can use a web
> interface, but you run the risk that with this design, you may
> never be able to pull the email later, POPStyle, into your
> computer. Although it is theoretically possible, using POP
> (rather than IMAP) to leave the mail on the server until you
> pull it again with POP, many servers appear to clear out the
> mail after POPing it. I think John Klensin made that
> observation in an earlier exchange.

I may not have, but, to get it on the table, many ISPs who offer
"free" POP3 accounts have decided that, in order to control
costs, they need to do one (or more) of three things:

	(i) Impose total-quantity-of-mail limits on mailboxes
	
	(ii) Charge for storage, at least above a certain,
	fairly small, amount.
	
	(iii) Prevent the user from using the mail store for
	long-term storage by deleting messages that have been
	downloaded, whether the user issues an explicit POP3
	"delete" command or not.

If the third model is adopted by a POP3 provider, then reading
through the mailbox looking for spam will presumably be 100%
effective in eliminating spam... because it will be 100%
effective in eliminating everything.  That is probably not the
desired result.

One of the other things you see to be handwaving a bit about is
the notion of handing out user IDs, passwords, and other
credentials to mail accounts to people so they can "help" with
spam (or other problems).  Sure, I can find a web-based
something-or-other to access my POP3 mailbox (if I had one).
But whether I can find one I trust as much as I trust my mail
store and the existing mechanisms for reaching it is an open
question.   If I understand what you are proposing --you have
actually been a little vague, and the web site listed in your
signature line seems to mostly run people around in circles--
you require that I supply you sufficient credentials to read
--and potentially delete and maybe alter-- my mail.    I assume
there are reasons why I should trust you that much, but,
independent of anything else, your establishing those would be,
I think, an interesting problem.

      john

---

From: Shelby Moore <[email-masked]>
Date: Mon, 08 Sep 2003 05:14:08 +0800

Keith, IMHO you started an excellent line for further debate (and not just because we have the same last name :).  It would be nice to see debate from both sides so that pros and cons could be fully explored.  I am not sure I am the one to carry the debate to extreme end (due to time constraints), but I will initiate the counter argument.  I came here to make a proposal.  It is not a proposal I would champion to the end game alone.  The feedback is valuable (hopefully to community-at-large as well), whether the proposal florishes or not.

>whatever) while everyone else can still use "push".

"Enforcers" will force the adoption.  "Enforcers" being Hosts, ISPs, legislatures, judiciaries, software, etc..

Once you define that *legitimate* bulk email is only delivered by "pull", then enforcers can start to filter bulk email.  This will force everyone to get their *legitimate* bulk email via "pull".  Else they will not get it.  If you have a choice between receiving your *legitimate* bulk email via "pull", or not receiving it, what are you going to do?

Now below you argue that enforcers will not enforce.  To me that is more viable argument, one that needs to be debated.  However, what is the harm in making an RFC and then find out if enforcers will enforce??????

more below...

>  And the question isn't

Agreed.  Good point about benefits (real results) of agreeing on definitions.  That is in essence why I am making this proposal.  I am proposing we make a definition of spam which is both agreed upon and not ambiguous to enforcers.  Simple as that.

In my mind, the thing to consider is what is the cost of making this definition compared to what is the benefit??  To me, that is one of the major aspects that need to be debated.  Economics (in the larger sense of "economy" as efficiency or survival/evolution, not just financial systems) *always* rules the outcome.

>the deployed user agents don't support it),

Most people already get their email through POP, even if there is a web-based client on top of it.  We already went through the discussion that Hotmail and Yahoo have the ability to access POP accounts.  They wouldn't have the feature if people aren't using it.

> most senders don't want the
>increased burden of providing POP/IMAP/NNTP/web servers

Only *legitimate* bulk email senders (mostly that is mailing lists) would have to do this.  Non-bulk email and spam does not change it's paradigm.  Spam is dealt with by enforcers.

Only *legitimate* bulk email senders (mostly that is mailing lists) would have to do this.

Only *legitimate* bulk email senders (mostly that is mailing lists) would have to do this.

Just making that very clear.

*Legitimate* bulk email senders already have a burden of maintaining mailing list databases, maintaining a subscribe and unsubscribe support, handling bounces, etc...s

> and the necessary
>customer support,

I assert there will be less or similar hassles with supporting a "pull" paradigm for *legitimate* bulk email than all the hasslings of teaching newbies how to subscribe and correctly interface with Majordomo mailing list.

I do not think it is a big difference.

And if you are protecting the mailing list administrators to save spam, then that seems like a bad set of priorities for internet email as a whole.

> and there are enough ISPs that derive significant revenue
>them all to enforce this.

This is wrong.  This would be saying that the economy of spam is significant compared to the internet economy as a whole. ISPs are losing $ big time to spam.

>  In short: nobody has sufficient incentive to adopt
>this.

Maybe "nobody" in this list, but let this thread sit in Google for a while and see what might snowball over time :)

>mail that people can use if they wish.

Great.  That is what I said above.  Why not see what the enforcers will do?  They can not do any thing now, because their hands are tied to ambiguous definition of spam as UBE.  They need spam == *BE in order to act.

>  If it works well, some people will

No.  If it works well (if the enforcers adopt), then everyone will use it, else they won't receive their *legitimate* bulk email.

---

From: Shelby Moore <[email-masked]>
Date: Mon, 08 Sep 2003 05:34:07 +0800

This is getting way off topic.

>spam (or other problems).

My proposal has nothing to do with IDs so let's just drop that line of discussion or respond with a different subject to move it to a different thread.

>  Sure, I can find a web-based

Agreed.

>the web site listed in your
>signature line seems to mostly run people around in circles

My web site and businesses have nothing to do with this proposal.

The proposal is to define that *legitimate* bulk email must be sent and received by "pull" instead of by "push".  So that all remaining bulk email is spam.  This changes spam from "Spam == UBE" (an ambiguous definition for enforcers) to "Spam == *BE" (UNambiguous definition for enforcers).

How *BE gets dealt with by enforcers is not related to my businesses or web site.  Enforcers is a widespread term meaning any one who will enforce the proposed RFC against spam.

FYI: (completely off topic so please don't respond to list on this point) The reason the web site runs in circles is it isn't finished yet, nor ready to be released to public at large yet.  It should be completed in a few more days.

Shelby Moore
http://AntiViotic

---

From: Keith Moore <[email-masked]>
Date: Sun, 7 Sep 2003 19:28:16 -0400

>  However, what is the harm in making an RFC and then find out if enforcers
>  will enforce??????

you appear to presume that you can get consensus support for such a plan
from within IETF.  even if you could get such support (which you cannot)
note that there's no enforcement of IETF's other opinions, even in cases where
failure to adhere to IETF standards costs billions of dollars every year.  why
would this case be any different?

there are a lot of ways to solve the spam problem that will work if "everybody
does it my way."  so far, nobody has figured out how to impose their will on
the rest of the net.

Keith

---

From: shogunx <[email-masked]>
Date: Sat, 6 Sep 2003 20:21:11 -0400 (EDT)

> so far, nobody has figured out how to impose their will on
> the rest of the net.

thankfully

---

From: Shelby Moore <[email-masked]>
Date: Mon, 08 Sep 2003 08:03:04 +0800

>from within IETF.

No, no.  I try to never beg.

I came here to make a public proposal and some points for the purposes of public record.  I never expected any one to really do any thing on this proposal now.  But by placing the idea into public domain now, as the rate of spam approaches the asymptote of 100%, then mailing list administrators can come back to this idea as a way to save themselves.  Actually at that point, they won't need the RFC or STD, because all bulk email will be blocked and they will be forced to just go straight to offering a "pull" solution (whether it be web, usenet, pop or whatever) so recievers can still get their mailing list messages amongst the 10,000 spams per day.

I am not sure exactly how long this will take, but without another solution or paradigm shift, I figure 2 - 5 years tops.

>  even if you could get such support (which you cannot)

How do survey their opinion??

>would this case be any different?

Because enforcers want to do something and will be under increasing pressure to do something against bulk email.

>does it my way."

I have seen none.  Email signing won't work.  No amount of <!plonks> from Randy Bush will change that.

Changing SMTP won't work long term, etc..

>  so far, nobody has figured out how to impose their will on

It is not my, your, or IETF will, but the votes of 500 million (through their actions against spam) that will impose.

Shelby Moore
http://AntiViotic.com

---

From: Vernon Schryver <[email-masked]>
Date: Sun, 7 Sep 2003 18:38:08 -0600 (MDT)

> From: Keith Moore <[email-masked]>

> note that there's no enforcement of IETF's other opinions, ...
> ...

yeah.

I've been compiling a list in the style of Jeff Foxworthy.

                You Might Be An Anti-Spam Kook If

   - you have discovered the Ultimate Final Perfect Solution To The
      Spam Problem (UFPSTTSP).

   - you are the first to think of the UFPSTTSP.

   - you were motivated to find the UFPSTTSP because you know it is
     impossible to filter more than 99% of spam with fewer than 0.1%
     false positives by any of several currently available mechanisms.

   - despite being the inventor of the UFPSTTSP, you are unfamiliar with
     "false positive," "false negative," "UBE," "tarpit," "teergrube,"
     "Brightmail," "Postini," "SpamAssassin," "DNS blacklist," "HELO,"
     "RBL," or "mail envelope."

   - you plan to make money by licensing the idea of the UFPSTTSP.

   - you are deeply offended when people do not agree that you have
      found the UFPSTTSP.

   - you cannot name several potentially fatal flaws in the UFPSTTSP.

   - you think all you need to do to get the UFPSTTSP implemented and
      deployed is to publish an RFC.

    - you don't recognize the difference between deploying and implemeting
      the UFPSTTSP.

   - you plan to publish an RFC mandating the UFPSTTSP but have no idea
      that RFC 2223 or RFC 2026 exist.

   - you have no idea of the relevance of "consensus" or "IESG approval"
       to publishing RFCs.

   - you think all RFCs have the same standing.

   - you think that spammers won't ignore, subvert, or exploit the
      UFPSTTSP if you publish it as an RFC.
       
   - the UFPSTTSP depends on spammers or mail recipients changing
      their behavior without any immediately gain.

   - the UFPSTTSP won't be effective until it has been deployed at more
      than 60% of SMTP servers and you don't think that's a problem.

   - the UFPSTTSP is trivial to implement and deploy, but you have
       done neither.

   - you feel your job is done after having explained the UFPSTTSP, and
      that "programmers" will drop everything to implement it.

   - you think that a violation of an RFC by an SMTP client or server
      is good and sufficient reason to reject all mail from the system's
      domain.

   - you think that SMTP has no authentication and have never heard of
      SMTP-AUTH, SMTP-TLS, S/MIME, or PGP or think they're irrelevant
      to the lack of authentication in SMTP.

   - you think that the fact that most SMTP servers do not authenticate
      the SMTP clients of strangers is a major bug in SMTP instead of
      a major feature and expression of a primary design goal.

   - despite discovering the UFPSTTSP, you don't know the meanings of
      MTA, MUA, SMTP server, or SMTP client.

   - the UFPSTTSP requires a small number of central servers that for
      validating email, serving as "pull servers" for bulk mail, or
      anything else.

   - the UFPSTTSP requires that anyone wanting to send mail obtain a
      certificate and that such certificates would be checked by
      all SMTP servers.

   - you think that useful certificates of a person's identity that
      certifies not only that the person has name but has no other
      certified names are cheap and easy.

   - you think that most Internet users would willingly pay more
      $5/month to avoid spam, and don't know the per-user price
      point for anti-virus software or data.

   - you don't see why a certificate that binds a name to a user is
      useless against spam unless it also certifies that the user has
      no other names.

   - the UFPSTTSP involves ISPs issuing certificates to users and that
      the same ISPs that don't terminate the accounts of spammers or
      don't investigate prospective customers enough to refuse service
      to spammers will refuse certificates to spammers.

   - you've never heard of RFC 2554 or RFC 2487 and the UFPSTTSP
      has something to do with authentication.

   - the UFPSTTSP involves replacing SMTP.

   - you routinely send single "LARTS" or reports of single examples
      of objectionable mail to more than two dozen addressees.

   - your definition of spam differs significantly from "unsolicited
      bulk email".

---

From: Shelby Moore <[email-masked]>
Date: Mon, 08 Sep 2003 08:50:53 +0800

Whew!  That is a long list, and fortunately I don't exhibit any of your stated symptoms of "Anti-Spam Kook" so I guess I should feel sane??

Too many to addresses all, but here are a few...

>   - you have discovered the Ultimate Final Perfect Solution To The

I am predicting spam to continue increase and in fact accelerate with either my proposal or my other work.

Face it, filtering of bulk email is going to happen and bulk email will increase because of it.

This might seem contradictory to other posts I have made, but it is not.

Whew! Thus no other UFPSTTSP  below apply to me.

>     false positives by any of several currently available mechanisms.

Whew!  Luckily, I know what a mathematic asymptote,  limit theorem, and confidence interval are.  I guess I am still sane, praise the lord!

>     "RBL," or "mail envelope."

And AccuSpam.com :)

>   - you plan to make money by licensing the idea of the UFPSTTSP.

I hate licensing and find sales to end users much more direct way to earn income, especially when you've already automated that process in your other businesses.

>   - you have no idea of the relevance of "consensus" or "IESG approval"

I know I started here:

http://www.faqs.org/rfcs/inet-standards.html

and here:

http://www.faqs.org/rfcs/editor-info.html

Which refers to all your points and many which you missed!

>      UFPSTTSP if you publish it as an RFC.

One immutable fact is spammers will ALWAYS have to send in bulk.

>      is good and sufficient reason to reject all mail from the system's

SOBO (statement of the blatantly obvious)

>      MTA, MUA, SMTP server, or SMTP client.

Yeah that would be bonehead.

>      validating email, serving as "pull servers" for bulk mail, or

Agreed centralized solutions do not work.

>      certificate and that such certificates would be checked by

Email signing is failed concept.

>      point for anti-virus software or data.

Free is nice for most.

>   - the UFPSTTSP involves replacing SMTP.

Morphing SMTP is a losing concept to anti-spam.

>      of objectionable mail to more than two dozen addressees.

Sending 24+ emails of anything might also qualify.

>   - your definition of spam differs significantly from "unsolicited

Hint: unless that definition also corresponds to what is subjectively "unsolicited bulk email" most of the time.

Nice fishing expedition..."you know what I mean Vern".

Shelby Moore
http://AntiViotic.com

---

From: Shelby Moore <[email-masked]>
Date: Mon, 08 Sep 2003 09:42:01 +0800

>     "Brightmail," ...

Another interesting tidbit is that Enrique Salem, the CEO of BrightMail (which is a Symantec company), is my former classmate from Culver City High School class of 1983.

http://www.brightmail.com/about_us-exec_team.html#enrique_salem

That has absolutely no relevance, as neither does this thread by Vernon.  BrightMail in my experience with Earthlink, filters 90+% of spam with very low false positive rate.

Shelby Moore
http://AntiViotic.com

---

From: [email-masked]
Date: Sun, 07 Sep 2003 23:32:04 -0400

>                 You Might Be An Anti-Spam Kook If

Man, you know how hard it is to get coca-cola off a laptop keyboard? ;)

(Fortunately, most of it missed the laptop.. ;)

>       useless against spam unless it also certifies that the user has

Actually, that one's kind of subtle, even I had to read it twice before I realized
that (a) almost all UFPSTTSP's pull a spectacular crash-n-burn on one of
the other criteria first, and (b) it's usually stated the other way around (left
as an exercise for the reader ;)

---

From: Vernon Schryver <[email-masked]>
Date: Sun, 7 Sep 2003 22:49:48 -0600 (MDT)

> From: [email-masked]

> as an exercise for the reader ;) 

I've tried to improve that one one as well as make other changes
suggested by various people in
http://www.rhyolite.com/anti-spam/you-might-be.html

I'd be glad to hear more suggestions, but they should probably
not involve this mailing list.

---

From: Shelby Moore <[email-masked]>
Date: Mon, 08 Sep 2003 13:04:48 +0800

>I've tried to improve that one

Vernon,

<rant>
What are you "improving"?  Oxymoron.

Would it be any thing related to Internet Engineering (as in IETF)?

Is this really appropriate posting for someone who apparently runs the DCC, which is apparently supposed to a professional service that millions of clients rely on?

Is this the kind of stuff you do and think about all day?  What about actually doing some real work on Internet Engineering?

Is this the kind of posting you want publicly distributed when later someone Googles or otherwise investigates what are the thought processes and professionalism of the people engineering the foundations of the internet.

Frankly, I will be linking back to this thread if any one ever asks me why spam can be solved at the internet engineering level.

I sense you feel that some people who have ideas do not know enough about what they are posting, but you might be surprised what people actually know.  I've searched your posts all over the web via Google groups and you have a history of being obstinate, condescending, and impolite.  You are also much less of an expert than you portend to be relatively speaking...
</rant>

Shelby Moore
http://AntiViotic.com

---

From: Zefram <[email-masked]>
Date: Mon, 8 Sep 2003 10:14:16 +0100

>                You Might Be An Anti-Spam Kook If

Please publish this as an RFC.  A collection of unworkable approaches
to the spam problem (anti-spam anti-patterns) is useful knowledge that
should be preserved and promulgated to reduce the Anti-Spam Kook problem.

-zefram
-- 
Andrew Main (Zefram) <[email-masked]>

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Mon, 8 Sep 2003 11:30:08 +0200

> should be preserved and promulgated to reduce the Anti-Spam Kook 

On the other hand there seems to be considerable interest into 
declaring the spam problem unsolvable. I don't think it's a good idea 
to lend credibility to this sentiment by publishing it as an RFC.

How hard is it to agree that:

a) there will always be (some) spam
b) there is no need for it to be 50% of all mail

---

From: Shelby Moore <[email-masked]>
Date: Mon, 08 Sep 2003 18:29:00 +0800

>You'd do well to observe and learn.

What specifically did you learn from Vernon in this thread?

---

From: Shelby Moore <[email-masked]>
Date: Mon, 08 Sep 2003 18:38:03 +0800

>to the spam problem...

Imagine if we engineered all things in real life by first expending effort publishing all the ways not to engineer them.

That would be a good way to insure that nothing would ever get engineered.  It would *stifle* all or most production.

In mailing lists I have observed there are usually a few key guys who are sort of like the local bullies on the neighborhood street corner.  Their whole purpose in life is stifle any other boy from coming into the neighborhood and "stealing" (e.g. giving more opportunity to) the local girls.   Luckily they tend to not be very smart, else they wouldn't be wasting their time sitting around the local neighborhood.

---

From: Sandy Wills <[email-masked]>
Date: Mon, 08 Sep 2003 07:30:27 -0400

Shelby Moore wrote:
> In mailing lists I have observed there are usually a few key guys
 > who are sort of like the local bullies on the neighborhood street
 > corner.  Their whole purpose in life is stifle any other boy from
 > coming into the neighborhood and "stealing" (e.g. giving more
 > opportunity to) the local girls.   Luckily they tend to not be
 > very smart, else they wouldn't be wasting their time sitting around
 > the local neighborhood.

    In SOME neighborhoods (I won't say "most"; I haven't SEEN "most"), 
there is a guy who handles trouble for everyone else.  Maybe he likes 
it, maybe he doesn't but does it anyway because he's good at it and it 
helps the neighborhood stay quiet and safe.  What you call him really 
depends upon your point of view.  Yes, troublemakers call him a bully. 
But, the quiet peaceful people who rely on him to keep their 
neighborhood quiet and peaceful while they go about their business think 
of him more as the policeman who happens to live on our block.

    Everyone has ideas.  Some good; some bad.  Please, bring your idea 
here, and we'll discuss it.  But, it's hard to have sympathy for your 
whining when you are told by _several_different_people_ that this idea 
has flaws.  Think them over, and improve your idea with the input you 
get from others who have been thinking about the same problem.  Bring 
back your improved idea, and we'll do the cycle again, until we either 
come up with a workable solution, or give up.

    I am usually just a lurker here, but this thread doesn't look like 
it is going anywhere useful, and it has lost it's entertainment value. 
In fact, while your posts can't really be considered spam, they are 
rapidly attaining the same value.  Please remember that anything you 
post to this discussion list will be archived, and available for the 
whole universe to see.
    Ask yourself "Will I be happy, 30 years from now, to know that my 
grandchildren are reading this post to find out what grandpa was like?"

-- 
: Unable to locate coffee.  Operator halted.
: Reply to [email-masked]

---

From: Shelby Moore <[email-masked]>
Date: Mon, 08 Sep 2003 19:25:22 +0800

>b) there is no need for it to be 50% of all mail

I agree.

>declaring the spam problem unsolvable.

Amazing that Vernon says working on spam is "Kook"y, yet that is exactly what he does (DCC).

Vernon has pretended he is an expert with this thread, with an obvious timing of condescending the serious anti-spam thread I started:

http://www1.ietf.org/mail-archive/ietf/Current/msg22035.html

Some disinteresting things Vernon emailed directly to me in past, which imho demonstrate he is not an expert and/or he qualifies for his own "Kook" test:

1. Vernon apparently got offended because I pointed out that he didn't realize that MD5 checksum on IPv4 was easily breakable via dictionary attack or that his use of it went his often public stated condescending policy of "do not implement half-solutions".

2. Vernon wrote, "...it is impossible, because no two people (or at least organizations) think the same streams of bulk mail are solicited and unsolicited.".  I can easily find 2 streams that 2 people/orgs can agree are solicited and unsolicited.  If you give me 2 streams and 2 people/org at random, then I can not guarantee every time (but eventually and never impossible), but he said "no" which is always false and certainly never impossible.  That just shows you the kind of errors he makes with his logic.

3. Vernon wrote, "In the last couple of months, more than one person has invented a super duper wonderful perfect foolproof mechanism of using IP addresses collected by the DCC.".  "Forgive me, but I doubt your scheme differs significantly from the others.".  Isn't the desire to discover part of what makes a scientist an expert?  When we stop discovering and learning, then we decay.

4. "A fundamental problem is that spam is unsolicited bulk mail, and not IP addresses...no mater how highly coorelated with spam.".  The first part is true, the second part is not because we don't live in 3 dimensions.  I will not tell you why beyond that, but I know why.

5. "The DCC is doing quite well, thank you very much, against the current plague, but the main event is to come.".  Wonder what he means by "main event is to come"?

6. "consider me a hopeless, useless, paranoid skeptic with an insurmountable not-invented-here syndrome, and stop sending me mail"

Shelby Moore
http://AntiViotic.com

---

From: Shelby Moore <[email-masked]>
Date: Mon, 08 Sep 2003 19:39:58 +0800

>  But, it's hard to have sympathy for your 
>whining

That is interesting interpretation of my winning a debate so completely that no one has been able to poke one logic hole in the serious anti- spam thread I started:

Instead I think you must be referring to the person who started this silly thread.

> when you are told by _several_different_people_ that this idea 
>has flaws.

Which have all been resolutely refuted and defeated ad nauseum by me.

>it is going anywhere useful,

Makes you kind of wonder why Vernon posted a silly, condescending thread doesn't it.  If you can't beat me with logical debate, then resort to other tactics...

> and it has lost it's entertainment value. 

That is an interesting interpretation of condescending.

>In fact, while your posts can't really be considered spam,

And yours, and Vernons and everybody else who got suckered in by the person who started this condescending thread which has no engineering value.

Shelby Moore
http://AntiViotic.com

---

From: grenville armitage <[email-masked]>
Date: Mon, 08 Sep 2003 15:44:52 +1000

Shelby Moore rants (re Vernon):
	[..]
> I've searched your posts all over the web via Google groups
> and you have a history of being obstinate, condescending, and impolite.

He's also sometimes funny and accurate. You'd do well to observe and learn.

gja

---

From: grenville armitage <[email-masked]>
Date: Tue, 09 Sep 2003 00:04:20 +1000

Shelby Moore wrote:
	[..]
> Their whole purpose in life is stifle any other boy
> from coming into the neighborhood and "stealing" (e.g. giving
> more opportunity to) the local girls.

Ah. The ever popular "IETFers-are-jealous-because-I'm-sexy" argument.

I think we just need someone to mention Hitler and this thread is done.

Oops.

gja

---

From: Vernon Schryver <[email-masked]>
Date: Mon, 8 Sep 2003 08:21:19 -0600 (MDT)

> ...
> 1. Vernon apparently got offended because I pointed out that he
> didn't realize that MD5 checksum on IPv4 was easily breakable via
> dictionary attack or that his use of it went his often public stated
> condescending policy of "do not implement half-solutions".

Perhaps Mr. Moore should recall my claim to archive mail.  
He wrote this:

< 3. Most importantly, is there any reasonable way to extract the
< original IP from the checksum?  I suppose the IPv4 address space
< is 4 billion.  How long does it take to run 4 billion MD5 hashs?
< If impractical, I might be able to work with your checksums in my
< database instead of storing IP addresses (might not be a such a
< bad idea for privacy reasons).  However, why did you use checksum
< on IP any way (seems to me a hacker can get the original IP using
< a brute force attack)?

I responded:

] The DCC databases contain only MD5 hashes.  If you know of a way to  
] reverse MD5 hashes other than a dictionary attack, you should publish
] it and get famous.  I'm not sure, but you may be agreeing with that
] throught.
] 
] When the DCC databases contained MD5 hashes of IP addresses, they were
] of IPv6 addresses.  Of course, those IPv6 addresses were related in
] the standard way to IPv4 addresses.  I've not timed MD5 on 128 bit
] values, but guess 100 usec/hash.  If that's right, you could build a
] 16 GByte dictionary in about 100 hours.

(I made an arithmetic error in the figuring the size of the dictionary.)

Mr. Moore came back with:

} http://www.faqs.org/rfcs/rfc1810.html
} 
} 1995 RFC claims 87 Mbps rate for MD5 in software.  Assuming Moore's
} Law (double speed every 18 months), then we get 9 years (6 x 18
} months) to 2004, thus 6*87Mbps in 2004.
} 
} 32 bit = 2 ^ 32 = 4 billion / 6*87 millions = 24 seconds.
} 
} So if you had 1% of that space, or 40 million IPs in your databases
} over time, then would take approx. 20 million minutes = 333,000
} hours = 15,000 days < 50 years to convert all MD5 back to IPv4s.
} 
} However an inverse table could be built if we had 4 GB * 128 bit
} of storage = 4 * 16 GB = 64 GB.  This would drop the time to probably < month.
} 
} Assuming I am interpreting the RFC correctly.
} 
} Note I read some where that 2 ^ 64 search space is required before
} hitting the duplicate space of MD5.

and later:

| >When the DCC databases contained MD5 hashes of IP addresses, they were
| >of IPv6 addresses.  Of course, those IPv6 addresses were related in   
| >the standard way to IPv4 addresses.  I've not timed MD5 on 128 bit
| >values, but guess 100 usec/hash.  If that's right, you could build a
| >16 GByte dictionary in about 100 hours.
| 
| Correct 16 GB, not the 64 GB I mistakenly wrote late last night.

Perhaps Mr. Moore's 16 GBytes comes from limiting the dictionary to
1 billion interesting IPv4 addresses.  Otherwise 64 GBytes is better.

I never did figure out what Mr. Moore meant by 15,000 days.  He
could not have been thinking of doing on average 2 billion MD5 hashes
for each of 4 billion IPv4 addresses, because that would have been
silly and would have take more than 15,000 days.

Ok, I'll stop feeding the troll now.

---

From: Randy Bush <[email-masked]>
Date: Mon, 8 Sep 2003 07:49:36 -0700

> Ok, I'll stop feeding the troll now.

troll?  what troll?  procmail is your friend.

randy

---

From: Dean Anderson <[email-masked]>
Date: Mon, 8 Sep 2003 11:30:54 -0400 (EDT)

> may be impossible, but that doesn't mean we have to resign to getting

People care because when someone comes around saying "you can have a 100%
impervious roof if only you jump through these inconvenient hoops", we
know that they are wrong, and don't need to waste time considering how
inconvenient the hoops are.

"We", meaning the IETF, care, because this is very useful aid to deciding
what to work on. We know that we need to focus on leak stoppage, not
trying to invent leak-proof protocols.  There is no point researching
something that is impossible.

> So when exactly was the earth supposed to stop moving?

God didn't make the earth move perpetually. He just made it move long
enough.  It seems that even God can't solve some problems.

We didn't get to the moon by inventing perpetual motion machines, though
early proposals were based on such machines.  We got to the moon by
working on the messy physics of rockets.

When someone comes to the NSF and says you can have a perpetual motion
machine if only you jump through some very inconvenient hoops, and spend a
lot of money, the NSF can save itself the time and money by discarding
perpetual motion schemes from its research program.  Similarly,
information theory allows us to discard some ideas from our research
programs.  That is why we care.

> Where does this covert channel stuff come from anyway?

What do you mean?

> > claim they are legitimate emailers, because they _ARE_ legitimate, so
> > far as we know before they send email. And even so far as we know
> > _before_ someone _READS_ their email.  Only after reading their email,
> > and perhaps only after some investigation, can we know for sure that
> > the sender and message is conducting abuse or in violation of their
> > AUP.
>

"Detecting abuse" is quite different from making a protocol that can't be
abused.  But that is my point: You have to focus on detection. This
doesn't require any protocol changes whatsover.

We are already "only allowing known bulk emailers". Unfortunately, that
doesn't prevent spam.  Indeed, it seems most of the spam isn't commercial:
Most of the spam seems to come from viruses, and isn't really selling
anything.  The viruses can use the credentials of the infected user.
That is "legitimate", until someone reading the email realizes its not and
complains. These send 40-50 messages per IP, and is hard to detect as
bulk. But when added up over a lot of IP addresses, is quite obviously
annoying.

> > It is not immune to spam, though it distributes spam and other
> > broadcast messages much more efficiently than typical email systems.
>
> Ouch! :-)
>

No, that's the point. It isn't _fixable_ with authentication.  It isn't
fixable at all.  It is only "fixed" when the spammer loses his account.
Then the spammer gets a new account.  So it isn't really fixed.  So we are
always going to be playing a game of whack-a-mole.  That cannot be avoided
by altering the protocol or the authentication scheme (information theory
proves this). So it is useful, then, to work on ways of detection, and
improve our whack-a-mole skills.  Altering protocols and authentication is
a waste of time.

		--Dean

---

From: [email-masked]
Date: Mon, 08 Sep 2003 12:24:54 -0400

On Mon, 08 Sep 2003 18:38:03 +0800, Shelby Moore said:

> Imagine if we engineered all things in real life by first expending effort
> publishing all the ways not to engineer them.

Umm.. We don't.  The first documented spam was sent over 25 years ago,
and you tuned in late.

How to design a bridge:

1) Build one out of dirt.  Realize that's a dam not a bridge, when it washes
away.  Make a note of it.

2) Build one out of rocks.  Find out the hard way that it works better if you
use mortar too.  Make a note of it.

3) Build one out of rocks and mortar.  Find out the hard way that setting the
piers on mud causes washouts.  Make a note of it.

4) Build one out of rocks and mortar, and set the piers on good solid rock.
Find out the hard way (multiple times) about how thick it has to be to support
a given load.  Make a note of it.

5) Discover the hard way about resonance effects.  Post a sign for the
marching troops and make a note of it.

6) Bundle up your notes and publish the Civil Engineering Handbook, First Edition.

7) Discover that wood makes OK bridges too, make lots of notes about strength,
insect prevention, and the like.  Publish the Second Edition.

8) Discover metal.  Make notes about fatigue and rust.  Publish Third Edition.

We can't help you if you think that we sat down and wrote the Third Edition
before doing anything else.  And we *really* can't help you if we hand you a
copy of the Third Edition, and you insist on trying to build a bridge in a way
that we pointed out on page 37 *just won't work*.  

> Amazing that Vernon says working on spam is "Kook"y, yet that is exactly what he does (DCC).

No.. What he *said* was "totally disregarding the things others have already tried is kooky".
Which line item from his note is the DCC in violation of?  (In particular, note that Vernon
has never claimed that the DCC is either perfect or a complete solution - it's a work-in-progress
of one tool for one aspect of the problem).

> 2. Vernon wrote, "...it is impossible, because no two people (or at least
> organizations) think the same streams of bulk mail are solicited and
> unsolicited.".  I can easily find 2 streams that 2 people/orgs can agree are
> solicited and unsolicited.  If you give me 2 streams and 2 people/org at
> random, then I can not guarantee every time (but eventually and never

Right. You can't guarantee *every time* for 2 people at random.  That's his point exactly.  

Any scheme that requires any 2 random people to come up with the same answer on
this judgment call *will* break with either a false positive or false negative for each
object in the stream there's a disagreement on.

> 4. "A fundamental problem is that spam is unsolicited bulk mail, and not IP
> addresses...no mater how highly coorelated with spam.".  The first part is
> true, the second part is not because we don't live in 3 dimensions.  I will not
> tell you why beyond that, but I know why.

Actually, 3 dimensions has absolutely nothing to do with it.  And the fact that you
think it does indicates that you really don't understand the difference there.

The real problem is that there exists a non-empty subset of the IP address
space for which the predicate '((sends spam) XOR (sends non-spam)) ' is zero,
even for an actual SMTP source.

---

From: Randy Bush <[email-masked]>
Date: Mon, 8 Sep 2003 09:37:44 -0700

> 7) Discover that wood makes OK bridges too, make lots of notes about
> strength, insect prevention, and the like.  Publish the Second Edition.
> 

many years ago, i saw, in the uk midlands, a (large outdoor)
museum which had the first metal bridge.  it was built as if it
had the same characteristics as wood, i.e., big dimensional iron
bars the size/shape of the wood beams.  later, they discovered the
ibeam.

randy

---

From: Shelby Moore <[email-masked]>
Date: Tue, 09 Sep 2003 00:41:48 +0800

>Umm.. We don't.

I believe that when I see more posts in real engineering threads than I see posts in this thread.

>No..

Yes.  For one thing Vernon doesn't understand the concept of dynamic set theory and entropy as it applies to mapping IPs to spam.  That statement he made convinced me neither he or you have fully explored all the possibilities.

Neither of you still haven't sent any links to prior art to prove your assertion that I am rehashing.  You are all hot air and no facts.

> What he *said* was "totally disregarding the things others have already 
>tried is kooky".

The problem is he thinks he knows what other people are proposing but he doesn't understand it well enough to know it is different that what has been tried already.  And I doubt either of you have any clue about the mathematical ways it is different.  So before you write a master set of rules, maybe you should listen and learn a little more.

>Which line item from his note is the DCC in violation of?

Not DCC, but him as creator of DCC.  The very hippocritical fact that he publishes such a document that is in violation of the core philosophy of the document, which is in essense "don't think you know everything, unless you do, and you probably don't".

Especially the last one he added, which he got from one Keith Moore emailed about me:

"you're certain none of the preceding apply to you or you see nothing wrong in some of the"

He is either "certain" or is saying he is a Kook.  He fell right into his own trap.

>Right. You can't guarantee *every time* for 2 people at random.  That's his 

He did not write "every time".  He wrote "no".  "no" means none or empty set.  Learn about set theory.

Also you missed the deeper point, but that is to be expected...

>object in the stream there's a disagreement on.

Again you obviously don't have the mathematical background or don't know how to apply it.

I am sure that many people said that nuclear bomb could not be created, that the earth was flat, etc...

>Actually, 3 dimensions has absolutely nothing to do with it.

You are way off.  You did not even realize I was talking about the lack of time dimension, so you very likely still have no clue what I really mean.

>  And the fact 
>that you

I doubt you understand how entropy applies to spam.  And I doubt any of your prior art has ever looked it from that perspective.

>even for an actual SMTP source.

You are still stuck in binary logic.  You seem to be very far behind where my thought process is on spam.

Shelby Moore
http://AntiViotic.com

---

From: Shelby Moore <[email-masked]>
Date: Tue, 09 Sep 2003 01:12:05 +0800

>> condescending policy of "do not implement half-solutions".

And the applicable "Kook" rule from Mr. Schryver's bible that started this thread is:

http://www1.ietf.org/mail-archive/ietf/Current/msg22092.html

"you are deeply offended when people do not agree that you have found the UFPSTTSP"

Which Mr. Schryver conveniently removed from his web site version (wonder why?):

>Perhaps Mr. Moore should recall my claim to archive mail.  

Perhaps Mr. Schryver should recall my claim to archive mail.

The point I made in #1 above is that Mr. Schryver had apparently never considered the question until I asked it.  And then when I pointed out to him he got very offended, which violates one of his own rules for being a "Kook":

I wrote:
>> Yes that is why I wondered why you bothered to hash the IP at all.
>> I think this is case where you voilated your own principle of not
>> implementing "half solutions".  There is no security or privacy benefit
>> to hashing the IPs (because a brute force dictionary attack can be
>> built in 100 hours) and there would be benefits to not hashing them
>> such as my algorithm.  Now I have to carry around a 16 GB dictionary
>> (lookup table).

He responded:
>
>I doubt you have figured out how we thought the IP address might be used.

I responded:
What you thought is irrelevant to facts at hand.  The fact is the IPs can be reasonably extracted thus logically it is reasonably useless to hash them.

>could not have been thinking of doing on average 2 billion MD5 hashes

Can you read?  I wrote "1% of the space".

>Ok, I'll stop feeding the troll now.

I will stop feeding the troll by posting in this thread now.  I predict he or his buddies won't stop posting in this thread.

Shelby Moore
http://AntiViotic.com

---

From: Shelby Moore <[email-masked]>
Date: Tue, 09 Sep 2003 01:53:50 +0800

>> > constructing a system free of covert channels, which information theory

But information theory also says you can optimize signal-to-noise ratio, but only if you know what the characteristics of your signal are.

Thus my whole motivation for an unambiguous definition (spam == all bulk email) along the channel and not just a definition at the end points (UBE).

>impervious roof if only you jump through these inconvenient hoops",

Who said 100%?????

Is the problem already reduced to an acceptable S/N ratio for the majority?

Is the current S/N ratio a problem for the majority?

> we
>know that they are wrong,

How can you know "they are wrong", even you did not even realize that no one was proposing a 100% solution in this thread??

>what to work on.

You decline work without even understanding what is being written about a new idea?  No one ever proposed 100% solution.  I challenge you to find one post where I wrote that.

> We know that we need to focus on leak stoppage, not
>trying to invent leak-proof protocols.

I proposed an way to improve leak stoppage, by defining the signal in the channel and not only at end points.  I never proposed a leak-proof protocol.  Underlying in what you are saying is you don't support new protocols, because you have a vested interest in  existing methods of detection.

>We didn't get to the moon by inventing perpetual motion machines,

And many people said it was impossible for us to get to the moon.

> though

What about the messy information theory of defining the spam signal for the channel and not just the end points, so that it can be studied and research in the channel and not only in the receiver's mind????

Have you not realized yet that profound point I am making???

>> Where does this covert channel stuff come from anyway?

The use of covert here probably meant that making anything 100% covert is impossible in information theory.

>> > the sender and message is conducting abuse or in violation of their

You exactly stated the problem.  We have not defined spam (the signal we need to sample) in terms of the channel.  If you define it as ALL BULK EMAIL, then you can actually start some messy science of getting to the moon on spam issue...

>doesn't require any protocol changes whatsover.

You can not measure a signal if you have not defined it.  That is a fundamental concept of information theory.  Spam is ambigous in the channel, unless you define spam == all bulk email.  How many dozens of times have I written that in this thread!

>doesn't prevent spam.

SOBO (statement of blatantly obvious) that you can't filter something if you can't define it.  That information theory 101.

>  Indeed, it seems most of the spam isn't commercial:
>bulk.

Viruses are a different signal and can be filtered as such, unless they contain no viral data.

>proves this). So it is useful, then, to work on ways of detection,

Agreed.  And you can not detect well if you can not define what you are detecting.

>improve our whack-a-mole skills.  Altering protocols and authentication is

Er... unless they improve our ability to detect.

Shelby Moore
http://AntiViotic.com

---

From: Dean Anderson <[email-masked]>
Date: Mon, 8 Sep 2003 16:49:20 -0400 (EDT)

On Tue, 9 Sep 2003, Shelby Moore wrote:
> But information theory also says you can optimize signal-to-noise ratio,
> but only if you know what the characteristics of your signal are.

It actually doesn't say that precisely. It says that you can transmit a
signal with an arbitrarilly low error rate at a speed below the channel
capacity.

The concrete task of altering the signal to noise ratio is accomplished by
enhancing the signal with a harmonic oscillator, so that it is stronger
than the noise.  This is then described as a set of differential equations
that can be optimized with Variational methods.  The limits of this
process are indicated by information theory, the nyquist theorem, etc.
If the channel isn't described by a fourier series, then the differential
equations may not be solvable, and it may be impossible to optimize its
signal to noise ratio. (Well, there are other mathematical methods, but
you get the point.) You are borrowing the concepts by metaphor, but the
concrete methods don't transfer well.

My point is not to discourage you from trying to stop spam, but to focus
your attention on detection, rather than protocol alteration.  It is
impossible to alter the protocol in any way that will force the spammer to
identify themselves a-priori as a spammer.

You could ask for spammers to cooperatively self-mark their messages.
But this hasn't been terribly productive.  It is also pointless to ask for
cooperative identification of non-spammers and identify spammers as those
not in the set of non-spammers. It may be too strong to say this is
pointless, as qsecretary and things that do this are fairly popular,
However, they don't quite solve the general case of things that send mail
but don't receive mail. And of course such schemes are fooled by disposal
accounts, which can respond until they are shut down.  If everyone used
qsecretary, spammers would simply alter their software to send responses.

So given a set of unmarked messages, some spam, some not-spam, the task is
to have a program mark them in the same way that a human would if a human
were reading the messages. Since humans have different definitions of
spam, it would be useful if the program could accept different definitions
as well.  This is the realm of content analysis.

> Thus my whole motivation for an unambiguous definition (spam == all bulk
> email) along the channel and not just a definition at the end points
> (UBE).

You may need a precise definition before you can begin implementation
(just like you need a definition of voltage, current, etc to begin
building a transmitter), but you do not need a precise definition to talk
about the theoretical aspects.  Spam could be defined as UCE, CE, UBE, or
BE.  I have also a more complete and detailed taxonomy of spam:

There are 3 types of email that we generally call spam:

Type 1: Bonafide Messaging with a real Commercial or non-profit(ie
political) purpose.  This also includes contraband (eg drugs) which is
illegal in some or all juridictions, so long as they intend to deliver the
illegal goods. This also includes solicited and unsolicited email, though
it may be useful to distinguish solicited as Type 1A and unsolicted as
Type 1B.

Type 2: Bonafide fraudulent activity. Someone is really trying to get your
money, but has no intentions of honoring their obligations to the purchase
contract.  This includes bonafide attempts at identify theft.  This is
already criminalized by mail fraud and wire fraud, and other laws
concerning fraud.

Type 3: Annoyance activity. This has no bonafide intention of getting
money or even personal information, even though at a casual glance it may
appear so.  Type 3 is broken into 2 subtypes:

Type 3A is a relatively harmless disgruntled person, who is not terribly
sophisticated in their abuse, or in hiding their tracks. This type can be
handled by warnings or account termination. Besides spam, this type is
also involved in small DOS attacks and other unsophisticated abuse.

Type 3B is frequently a career criminal using viruses and rooted machines
to conduct annoyance, which is frequently just another type of DOS attack,
but targeted perhaps at an email address, or perhaps at a domain. This
type of attacker is frequently already a career criminal, having broken
into many, often hundreds of computers, illegally. This type cannot be
dealt with effectively by ISPs, because they are reasonable adept at
partially hiding their tracks by crossing organizational boundaries.
Usually, the ISP only detects the infected computer, but does not identify
or catch the cracker.

---

From: Shelby Moore <[email-masked]>
Date: Tue, 09 Sep 2003 10:39:36 +0800

After this issue, I am probably moving the thread to IRTF (as suggested) if possible (but probably after taking a break to do some other work).

>than the noise.

Agreed.

And thus on a "conceptual level", you have to have some idea about the signal characteristics in order to enhance it.

Actually if I remember correctly, your example is how it applies to periodic signals.  The general case is more abstract.

>  This is then described as a set of differential equations

Add Shannon entropy, chaos, etc...

>you get the point.)

Yes that is what I meant that the general case is more abstract, so I was talking on a "conceptual" or abstract level.

> You are borrowing the concepts by metaphor, but the

I was only using it to say we must define the signal how it appears in the channel before we can do any research on it in the channel.

The way spam is currently defined defined as UBE (instead of my proposed *BE), then it means you can only model the signal at the end point.  Given that means in the receivers subjective mind, that is not all that useful for research, unless you want to get into very fuzzy science such pyschology.  If you want to make the point about practicality, then that is a very strong one!

>My point is not to discourage you from trying to stop spam,

You are only 1 of 3 people so far at IETF who has said that to me.  The rest who have commented have tried to discourage me.  So thank you.

> but to focus

Disagree strongly.  First benefit is once you define spam == *BE (instead of UBE), then it is easier to model spam and do research on it, because you can model it at any node in the channel, not only at the receiver end point.  That was my whole point about "enforcers".

However, there is a problem.  Some *BE is solicited.  Which is why I proposed moving the solicited *BE to another channel ("pull").

Your point is that it is futile to define a protocol that will separate the solicited from the unsolicited, because spammers will always be able to subvert the protocol.  And you to say thus there are no benefits to detection.  I strongly disagree.  There are two aspects to my response:

1. Spam coming thru the alternate "pull" channel can be modeled differently that spam defined as *BE.  This separation of models provides benefits over trying to model spam as UBE in the receiver's mind (end point).  Other person in this thread has provided one specific example, which is the "pull" delay gives a whole new dynamic to detection.  Also I have pointed about that the membership quality of the solicited channel, gives it unique modeling advantages.

2. Spam coming thru the existing channel can then be modeled as *BE at any node of the channel, instead of as UBE.  Some nodes have a much better model of spam in this definition, than the one at the end point.  For example, ISPs can see a lot more abuse data in real-time, than a single receiver or the current inherently more clumsy attempts to group or poll receivers.

Hopefully that will set the record straight that I am thinking about spam in new conceptual ways...and not rehashing as others have claimed...

>But this hasn't been terribly productive.

Obviously I am not asking for that or any thing like that.  See above.

>  It is also pointless to ask for
>not in the set of non-spammers.

I am also not asking for this, and it is instructive to understand how I am not.

I am only making a definition, so that one can model under the benefits of that definition.  What people actually do is a different matter, but as I pointed out previously in this thread, once you model spam the way I have proposed, then solicited *BE will have a distinct advantage to adopt the model.  And as I point out above, it doesn't matter what spammers do, because the improved model is helpful for advancing detection in both cases.

And my other point has been that when a channel gets so saturated with noise that you can not longer find the original signal reliably (as you say above the S/N ratio will depend on Nyquist, which is a very crucial point), then solicited *BE and receivers are going to need a different model, else information transmission will no longer occur reliably.

>as well.  This is the realm of content analysis.

You see this is the crux of the whole stagnation of anti-spam in my view.  Content has nothing to do with what makes spam annoying.  It is the S/N factor, i.e. that it only gets a 0.005% response rate.

I am trying to shift the whole paradigm from thinking about psychology (will always be fuzzy result), to thinking and modeling the noise factor.

It is a profound paradigm shift that gets you closer to a more robust solution for detection.

>building a transmitter),

Exactly.  You need a definition before you can model.

> but you do not need a precise definition to talk
>about the theoretical aspects.

Yes you do.

>  Spam could be defined as UCE, CE, UBE, or

Those are all definitions.

>There are 3 types of email that we generally call spam:

This is going down into the psychology line of model, which I am trying to paradigm shift away from, because it is not very well correlated to what makes spam a problem.  If spam had a 5% response rate, it would no longer be a problem.  Modeling the psychology is something other people are working on already.

[snip]

Thanks,
Shelby Moore
http://AntiViotic.com

---

From: jfcm <[email-masked]>
Date: Tue, 09 Sep 2003 13:41:15 +0200

At 01:28 08/09/03, Keith Moore wrote:
>there are a lot of ways to solve the spam problem that will work if 
>"everybody does it my way."  so far, nobody has figured out how to impose 
>their will on the rest of the net.

True.

As you may know Intlnet is a old non-prof oriented toward community network 
support. I though we could publish drafts documenting our proposed 
solutions. I found we cannot foot the cost of such worthless debates as you 
note it. So we will try to gather all our needs (DNS, IPv6, mail, security, 
etc.) and used words and concepts into a one single "Intlnet-needs 
information draft". We hope this way we might help presenting worked on 
users demands, and worked out yet inexpert users solutions. In return we 
hope we will get expert comments from IETF.

We have a major spam problem and a solution I wish to discuss here as it 
fits the mission of no IETF group as I understand it (this for Harald).

First for you to understand, we use the word "ULD" for "User Level Domain". 
This is a community upper (TLD or SLD) level domain. This helps making the 
concept more generic. An ULD is defined as a community space of naming, 
trust, exchanges and services. It can be designed as a DNS "virtual zone". 
A virtual zone results user defined common characteristics used in a DNS 
file generator. The DNS files are then set-up to support it. So an ULD may 
span several layers or TLD or be a restriction of a zone.

One of our Members is to develop an ULD project implying millions of mail 
addresses in a primary spam target thematic area. They do not want to buy 
thousands of terra byte mail severs with huge bandwidth to filter or carry 
an expected 90/95% spam traffic.

So we devised weemail, I would thank you to review.

"Mail" was defined by a Programing Software Note 49 of Louis Pouzin end of 
1964 and first programmed by Tom Van Vleck. He did it in adding text (first 
on top of) to the recipient mailbox understand as a file. It followed the 
postal model, Louis is very keen of (datagram, zones). I come from a 
different experience and asked Tom and Louis why they did not just transfer 
indexes as I did when developing my own solution 20 years after for a QNX 
based network OS. Louis asked by return if I meant "URL" today? and Tom 
said he developed the pointer concept, that it worked the same.He 
documented the same problems I met. But on the systems he then used 
(Tandem) the needed demon was a complex issue (while it was a QNX adman was 
simple).

So we started discussing the ways, the cons and pros. We are now looking 
for a large provider to help the development phase, run it experimentally 
and then commercially.

1. we name it "web-e-mail" or "wee mail", so weemail. The 
http://weemail.org site is just in French right now. By lack of English 
competence? We should also update the information there.

2. The SMTP protocol is fully respected. There is NO change whatsoever in 
the network itself. Only possible modules to be added. We may want to speed 
some processes in telling in the header that a mail is a weemail. We think 
of a Mail-ID format not to change SMTP.

3. The mail text is only a weemail header (weeheader) made of the frist few 
lines, all starting with "weeXXX:" where XXX is a parameter freely defined 
at ULD level. Everything more is supposed to be stripped off somewhere on 
the path. No attachment, no worms.

4. The basic idea is to send "weeURL: xxx://abc.def.gh?cgi...=zzz" as a 
main or only weeheader information. Some more information of class 
(community of senders) or groups (community of sendees), classing, 
priority, language, type of information, protection and intensification 
etc. can also be passed in the weeheader.

5. This means that the only modifications are at the mail agent. To send a 
weemail it sends the weeheader and stores the message at the URL. To read a 
weemail (if the sendee wants it) it retrieves the messages at the URL.

6. As noted this kind of solution may dramatically increase the number of 
spam attempts. But:

     - the message must be actually stored at an identified URL what is 
more complex and compromising than sending an SMTP burst. We advocate an 
IPv6 address concatenating routing and addressing addresses with a single 
Host ID per host, facilitating the tracking of the Spaming hosts.

     - determining what is spam is a human decision. To track trapped 
copies may be complex. We think that a spammer needs to send 5 times more 
messages if the text stays only one hour in line (delay of mail propagation 
and short time for an equivalent population to retrieve it). This will make 
spam more costly and more noticeable. But also it will make spam far more 
easily identified though its weeURL an information easy to get and track. 
We expect spammers to harp on URLs, but they will need to keep a pattern 
(Domain name, IP, etc.) permitting to identify and report them. We intend 
to experiment to that end two new propositions : OPES for filtering and 
reacting and RID-DOS for inter operator real time alarms and reacting..

    - as you note - we need to impose a solution - to impose it on a 
consistent protected space. To that end only weemail will be accepted in 
every ULD mailboxes. Weemails will have to initiate from ULD dolman names 
and concern ULD weeURLs or partnering URLs..

7. The development as we see it consists first in a prototype mail agent 
and in proposing the acquired experience to main agent developers. Then we 
need to update MTA with a weemail module stripping everything from the 
weemails except the weeheader. Then we need to give an URL to every mailbox 
(this is a policy of the ULD to replace Whois by the web and get an 
universal directory). Eventually we expect to use weemail servers made of a 
webservice dialoguing with the mail agents. The last step could be to 
sometimes use OPES and OCP as a fast alternative to SMTP when an end to end 
service can be provided.

8. as users we see several additional netix advantages (we name "netix" the 
user network operating system like management). We can freely use any 
document format and intensification system with more flexibility than IRM 
and be compatible with many other architectures through simple message 
passing solutions (weeENTRIES in the header). The mail-agent/weemail server 
relation might be easily extended to support extended (global community) 
services. May be all could be made OPES directly running OCP.

jfc

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 9 Sep 2003 18:32:07 +0200

> inconvenient the hoops are.

Your analogy doesn't fly. Our email protocols have holes big enough to 
drive a truck through. Is it unreasonable when people ask the IETF 
leadership for a place to work on this?

> something that is impossible.

Let's first define our goal before declaring it impossible to reach.

> What do you mean?

The jump from "spam" to "covert channel" isn't immediately obvious. And 
if any proof of why spam is a covert channel has been offered, I've 
managed to miss it.

>> the spammer's achilles heel

> abused.

If you can detect abuse on input rather than on output, detecting abuse 
is exactly what enables you to make an abuse-free protocol. And again: 
just because there are situations that you can't do something doesn't 
mean it's automatically useless to perform the action under other 
circumstances too.

> But that is my point: You have to focus on detection. This

Do you mean first accepting the message, then searching for anatomical, 
pharmaceutical or financial terms and subsequently discarding the 
message? This is useless as it only makes the spammers spam more, not 
less.

> We are already "only allowing known bulk emailers".

Untrue.

> Unfortunately, that doesn't prevent spam.

It should help if used together with some kind of authentication (for 
which, I'm happy to see, there are several independently submitted 
proposals on the table) so that spammers can't inject arbitrary from 
addresses.

> anything.

The point being?

> The viruses can use the credentials of the infected user. That is 
> "legitimate", until someone reading the email realizes its not and

Not as annoying as "real" spam. (Although the size of email worms is 
unpleasant. Fortunately, regular spam is usually pretty small.)

> Then the spammer gets a new account.  So it isn't really fixed.

It is when they lose their accounts faster than they can acquire new 
ones.

> So we are always going to be playing a game of whack-a-mole.

So? I don't mind doing that if I can win. Having to play and then be 
forced to lose is much harder to swallow.

> That cannot be avoided

I find it unacceptable that my email software lets people bombard me 
with crap while at the same time faking the headers such that said crap 
seems to come from an innocent bystander. I can't understand how any 
reasonable person can think this is ok.

Or maybe we should just go ahead and move the "From: " header to 
historic and be done with it.

---

From: Vernon Schryver <[email-masked]>
Date: Tue, 9 Sep 2003 10:59:08 -0600 (MDT)

> > complains. These send 40-50 messages per IP, and is hard to detect as

Reports from some operators of DCC clients at non-trivial sites
claim that the DCC does a tolerable job against SoBig.F.  This is
without the Greylist support now available in the DCC client code.

The DCC detects bulk mail, defined as substantially identical
messages from any SMTP client senders.  I'd not expect the DCC to do
well against most worms or viruses.  SoBig is somewhat different.
(I won't talk those differences in public or with people I don't know
well enough to say they'll also be descrete.  Like other people who
care more about fighting viruses and spam than being known as fighters
of viruses and spam, I think the profit in idle chatter is not worth
the cost of giving even trivial aid and comfort to the bad guys.)

As has been pointed out, all of this belongs in the ASRG mailing list
if anywhere.

See http://irtf.org/charters/asrg.html 
and
https://www1.ietf.org/mail-archive/working-groups/asrg/current/maillist.html

---

From: Dean Anderson <[email-masked]>
Date: Tue, 9 Sep 2003 13:41:33 -0400 (EDT)

My apologies for this message.  This discussion is winding down. Iljitsch
makes some interesting points, to which I have tried to respond
thoughtfully.

		--Dean

> leadership for a place to work on this?

I don't think our email protocols have any holes at all. They can be
abused. But mere abuse is not a "hole".

> > "We", meaning the IETF, care, because this is very useful aid to
> > deciding what to work on. We know that we need to focus on leak
> > stoppage, not trying to invent leak-proof protocols.  There is no
> > point researching something that is impossible.
>

Well, I think the goal has been stated: Create an abuse-free email
protocol.  That goal is impossible. Thus, we have abusable protocols.

Perhaps you have a different goal in mind, but it doesn't sound like you
accept the premise that it impossible to create an abuse-free protocol.

> >>> After I first posted this on IETF a while back, someone suggested
> >>> that covert channels require cooperation, and that spam therefore
> >>> isn't a covert channel.
>

The NCSC's definition refers to ANY communication not authorized by the
security model.  Note that the term "Covert Channel" is frequently
associated with Multilevel Secure Operating Systems. The liturature uses
other terms to describe the same concept: "information leakage", "sneaky
signalling", "hidden data flows", "side channels". So you must not get too
caught up in the peculiarities of operating systems.  The concept is quite
general.

It is immediately obvious that covert channels have 3 characteristics: It
is a CHANNEL, it is COVERT, and it VIOLATES SECURITY POLICY. (Caps for
emphasis of definition, not loudness.)

CHANNEL:  Spam is a type of Email. Email is a channel transfering signals
from A to B. Email is really a subchannel of the internet, which is a
subchannel of the PSTN, public and private fiber networks, etc.

COVERT: Spam is hidden in so far as possible from the system operators. It
is an unintended communication in that the system operators intended that
only non-broadcast, solicited commercial communication flow. This not an
exclusive list of what is permitted, but illustrates that spam isn't
permitted.

VIOLATES SECURITY POLICY:  System Operators specified an acceptable use
policy that outlines what is permitted and what is not permitted. UCE is
not permitted.  Various methods have been implemented to enforce this
policy.

> >> the spammer's achilles heel is that they need to send out incredible
> >> amounts of email in order to fulfill their objectives, whichever
> >> those are. Detecting bulk mail is doable, and it shouldn't be too
> >> hard to come up with something to differentiate legitimate bulk
> >> emailing from spam. For instance, we can reverse the burden of proof
> >> here and only allow know bulk emailers.
>
> > "Detecting abuse" is quite different from making a protocol that can't
> > be abused.
>

Input to the first mailserver is output to the mail client.  There may be
many input/output connections.  For every input, there is an equal and
opposite output. (Seems that perpetual motion may have some analog after
all ;-)

> > We are already "only allowing known bulk emailers".

Speak for yourself.

> proposals on the table) so that spammers can't inject arbitrary from

Spammers can _always_ do whatever regular users can do. There is no way to
authenticate regular users and not authenticate spammers. This is why SMTP
AUTH has no effect on spam.  If regular users can inject arbitrary
addresses, then so can spammers.  Regular users can always inject
arbitrary addresses.  Ordinarilly, this is desired.

> > Indeed, it seems most of the spam isn't commercial: Most of the spam
> > seems to come from viruses, and isn't really selling anything.
>
> The point being?

The point being that the persons sending most of the spam will do anything
to annoy people. It has no grounding in commercial motivation, nor can
regulation of commercial activity have any effect. In other words,
protocols that require cooperation of the spammer aren't likely to
succeed.

> unpleasant. Fortunately, regular spam is usually pretty small.)

You miss the point. Most of the "little" spam is sent by virus infected
computers as well as email containing the virus itself.

> It is when they lose their accounts faster than they can acquire new

Has this ever happened in 8+ years?  No.  Will it ever happen? No: ISPs
aren't ever going to terminate accounts immediately on someone elses
say-so.  First, it wouldn't be good business not to investigate
accusations, second, it wouldn't be legal, and third, the blacklists have
a terrible record of abuse, themselves.

The spammer already has authentication, however it was acquired: Stolen,
disposable, etc. Email authentication will always be equivalent to that
authentication. The user either has both, or they have neither. If they
lose their account, they lose both.  It is unnecessary to add more
authentication.  An engineer should know that two variables that always
have the same value can be replaced by one variable.

The notion of authentication in email comes from the fallacy that spammers
are somehow outsiders getting in.  They are not outsiders. They will never
be outsiders. It is impossible to construct a protocol that makes them
outsiders, any more than it is possible to make a protocol that keeps out,
say, democrats.

> forced to lose is much harder to swallow.

You can only win by detection and suppression.  Suppression has to happen
faster than account termination. The only way that can happen is by
filtering the recievers email.

> > That cannot be avoided by altering the protocol or the authentication
> > scheme (information theory proves this). So it is useful, then, to
> > work on ways of detection, and improve our whack-a-mole skills.
> > Altering protocols and authentication is a waste of time.
>

People find it unacceptable that fertilizer explodes.  However, it is
impossible to make fertilizer that doesn't explode. Only those ignorant of
chemistry call for such things.

> historic and be done with it.

Might as well add all the other headers than can be forged as well. Might
as well take the From Address off postal mail as well, since that can be
forged, too.  It is a mistake to think that just because something can be
abused, that it should be removed.

		--Dean

---

From: Bill Strahm <[email-masked]>
Date: Tue, 9 Sep 2003 13:44:25 -0400

Why is this even difficult.  I have yet to see a firm proposal (ie. an
Internet Draft), and once there is one, it is a simple matter of asking an AD
to sponsor a BOF to see if there is interest in forming a working group to 
solve the problem.  I remember sitting through several YATP (Yet another
Tunnelling Protocol) BOFs years ago, I don't see what the problem is with
creating some YASPP BOFs (Yet Another Spam Prevention Protocol).

This is the IETF, that is the IETF process... Why are we arguing here about
killing it without having a firm proposal, and a BOF chartered to form a WG
to go solve the problem.  Any more breath we waste now doesn't help anybody.

My challenge - Go forth - publish your protocol in ID form, contact the 
correct APP (probably) area AD and get a BOF in Minneapolis - Convince the
IETF in general there that a WG should be chartered to solve the problem.
Go and solve the problem

Bill

---

From: Shelby Moore <[email-masked]>
Date: Wed, 10 Sep 2003 04:45:06 +0800

>makes some interesting points, to which I have tried to respond

Dean,

Yes as already stated, I do intend to close this thread and eventually provide a forwarding link to a new discussion elsewhere (perhaps at IRTF as someone suggested, if possible...)

However, I think the analysis of the concepts of information theory, channels, and models of spam is more fundamental to "internet engineering" than the original purpose of this thread and thus I see no reason why it would not be useful data here at IETF.

Before I respond to your continuance of your argument, I *respectfully* remind that I already refuted the whole line of criticism you are continuing in this post, when I rebutted your last post in this thread:

http://www1.ietf.org/mail-archive/ietf/Current/msg22139.html

In case any one missed it, I think the most relevant section there begins done the page a bit with:

"Disagree strongly. First benefit is once you define spam == *BE (instead of UBE), then it is easier to model spam..."

more below...

>abused. But mere abuse is not a "hole".

Semantics debate only.  Better to stick to the real point below...

>protocol.

No that is not the stated goal of this thread I started. I already rebutted that whole link of criticism here:

http://www1.ietf.org/mail-archive/ietf/Current/msg22139.html

Look for the section that starts with:
"Your point is that it is futile to define a protocol..."

And here:

http://www1.ietf.org/mail-archive/ietf/Current/msg22129.html

Start reading down from:
"I proposed an way to improve leak stoppage, by defining the signal in the channel and not only at end points. I never proposed a leak-proof protocol."

>accept the premise that it impossible to create an abuse-free protocol.

The links to the previous posts are above which state that is not our goal.  You have been told that at least 2 or 3 times already.

>> Iljitsch van Beijnum wrote:

Iljitsch van Beijnum, I think what Dean Anderson means is that because you can't create a 100% perfect covert channel, then spammers will always find a way to abuse, no matter what you do on the protocol level.  Theoretically I agree with him.  However, he is ignoring the posts I made (as linked above), which show that is not what I am proposing.  What I am proposing has to do with improving the model of spam so it can be more easily detected at more points in the channels and earlier and other detection advantages.  To get this model, I propose that we need a new definition of legitimate bulk email, from "push" to "pull"....rather than repeat my entire logic here, please read the linked posts above in entirety.

> Dean Anderson wrote:

And COVERT has nothing to do with my proposal as I've detailed ad nauseum in the above linked posts.

>subchannel of the PSTN, public and private fiber networks, etc.

And moving legitimate bulk email to a "pull" channel is part of my proposal.

>exclusive list of what is permitted, but illustrates that spam isn't

Part of my overall point made in the links of posts above is that one of the reasons it is "hidden" is because it can only currently be modeled pyschologically, because the definition is UBE (unsolicited bulk email) instead of *BE (all bulk email).

Again read the linked posts above more carefully.  With a different model of spam, we aren't stopping abuse, we are merely increasing detection by having a better model of the signal.

>> > Dean Anderson wrote:

This thread is not proposing that.  See above.

>> Iljitsch van Beijnum wrote:
>> If you can detect abuse on input rather than on output,

Correct that is the point of improving the model of the spam signal, so we can do things at earlier points in the channel, input to mailing lists, input to dialup accounts, ISPs, Hosts, etc.

Right now, ISPs and Hosts can do nothing because they can not say that all bulk email is spam, therefor they can not be proactive in real-time.  That is just one example of many benefits to improving the model the way I have proposed.

>> Iljitsch van Beijnum wrote:
>> detecting abuse
>> is exactly what enables you to make an abuse-free protocol.

No we can not get a 100% abuse free protocol.  Information theory tells you that is impossible and I agree with Dean on that.

But we can get a better model which helps us detect more abuse.

>Spammers can _always_ do whatever regular users can do.

Mostly yes.  And that is why improving the model of the spam signal is the only real way we are going to get better at detection.  Actually your theories are making my proposal stronger vs all the other ways of detecting spam.  My point for the last 2 years has been that any model of spam which looks at content is going to fail over the long-term because content is what legitimate mail does also.

So you have to look at modeling what is actually different about the spam signal.  I will make a very profound AXIOM based on information theory:

AXIOM 1: The only way to reliably detect spam over the long-term is by modeling that which is unique to spam signal and not shared with legit email signal.  And that is the "bulkness" of it and/or the low response (or read) rate.

Now I wrote "mostly yes" above because once you move legitimate bulk email to a "pull" channel, then as Iljitsch van Beijnum originally pointed out, you can authenticate spammers to subscribed lists differently than legit users of that list.  In email channel the signatures of spam are bulk and low response rate.  In a subscribed  "pull" channel, the spammer's signature is again what??  Think about that deeply before you try to rebutt it.  (Hint: no response rate and the fact that messages which are responded to can be automatically deleted from the pull queue before many users might pull them...and other possible algorithms)

Shelby Moore
http://AntiViotic.com

---

From: "Spencer Dawkins" <[email-masked]>
Date: Tue, 9 Sep 2003 16:30:55 -0500

Oh, please tell me you're not going to keep posting pointers to your
previous postings until everyone agrees with you.

Spencer

----- Original Message ----- 
From: "Shelby Moore" <[email-masked]>

[deleted down to ]

>
> Before I respond to your continuance of your argument, I
*respectfully* remind that I already refuted the whole line of
criticism you are continuing in this post, when I rebutted your last
post in this thread:
> In case any one missed it,

[deleted down to]

>
>
> No that is not the stated goal of this thread I started. I already
> http://www1.ietf.org/mail-archive/ietf/Current/msg22129.html

[deleted down to]

>
>
> The links to the previous posts are above which state that is not
our goal.  You have been told that at least 2 or 3 times already.

[deleted down to ]

"pull"....rather than repeat my entire logic here, please read the
linked posts above in entirety.

[deleted down to]

> And COVERT has nothing to do with my proposal as I've detailed ad
nauseum in the above linked posts.

[deleted down to ]

>
> Again read the linked posts above more carefully.  With a different
model of spam, we aren't stopping abuse, we are merely increasing
detection by having a better model of the signal.

[deleted down to]

---

From: Shelby Moore <[email-masked]>
Date: Wed, 10 Sep 2003 05:10:06 +0800

>Internet Draft),...
>My challenge - Go forth - publish your protocol in ID form...

1. I remind you to read my initial post that started this thread:

"Request for opinions on whether to creating a working group or publish the following idea as an internet draft?"

I think that qualifes under "initial discussion" of the charter of this list (see #2 below).

2. And to read the charter for this mailing list:

http://www.ietf.org/maillist.html

"This list is meant for initial discussion only. Discussions that fall within the area of any working group or well established list should be moved to such more specific forum..."

3. Just a fews posts back in this thread, it was suggested that IRTF would be a better forum for anti-spam proposals and discussions, and I agreed (to consider it if possible and applicable).

However there is a another line of discussion in this thread pertaining to general information theory and modeling of channels which is still winding down ("initial discussion") and seems quite general to internet engineering.

4. The basic difficulty has been those violating the charter of the list:

http://www.ietf.org/maillist.html

"Unprofessional commentary, regardless of the general subject." such as the "Kook" thread offshoot that someone started.

Shelby Moore
http://AntiViotic.com

---

From: Shelby Moore <[email-masked]>
Date: Wed, 10 Sep 2003 05:21:24 +0800

>claim that the DCC does a tolerable job against SoBig.F.
....
>...  I'd not expect the DCC to do
>well against most worms or viruses.

I agree in that it seems to me on an internet engineering level of analysis, it makes a lot more sense to remove the viral attachments BEFORE the MUA.  Afaik, the DCC (usually) works at the MTA layer which is nice for system admins but perhaps not ideal for users that don't have system admins.

If we comparing solutions to viral attachments (that also do anti-spam), ours operates after the MTA but before (statistically) the MUA, and is instantly user configurable:

http://AntiViotic.com/test.php

Will solve 100% your viral attachment problem with the following advantages:

http://AntiViotic.com/antiviotic.php

Shelby Moore
http://AntiViotic.com

---

From: Shelby Moore <[email-masked]>
Date: Wed, 10 Sep 2003 05:32:25 +0800

>previous postings until everyone agrees with you.

If Dave Anderson (or any one else) keeps making new posts *misstating* that my proposal is to make an abuse-free protocol, when my proposal is not, then I guess I might have to keep clarifying that my proposal is not to make an abuse-free protocol, eh?

I would help to actually read the post before replying, in order to cut down the noise level.

Shelby Moore
http://AntiViotic.com

---

From: "Spencer Dawkins" <[email-masked]>
Date: Tue, 9 Sep 2003 17:02:03 -0500

> If Dave Anderson (or any one else) keeps making new posts
*misstating* that my proposal is to make an abuse-free protocol, when
my proposal is not, then I guess I might have to keep clarifying that
my proposal is not to make an abuse-free protocol, eh?

Apparently Randy Bush is unavailable, so I will repeat his previous
post ...

> troll?  what troll?  procmail is your friend.

Spencer (with apologies to Randy, but I bet he isn't too upset)

http://www1.ietf.org/mail-archive/ietf/Current/msg22117.html

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 09 Sep 2003 15:03:02 -0700

Note that advertising is inappropriate on the IETF mailing list.

--On 10. september 2003 05:21 +0800 Shelby Moore <[email-masked]> 
wrote:

> If we comparing solutions to viral attachments (that also do anti-spam),
> ours operates after the MTA but before (statistically) the MUA, and is
> instantly user configurable:

---

From: Shelby Moore <[email-masked]>
Date: Wed, 10 Sep 2003 05:50:18 +0800

>Note that advertising is inappropriate on the IETF mailing list.

I don't think that was Vernon's or my intention.

Then remove both Vernon's (DCC) and my post.  I saw it as implemented examples of two different fundamental internet engineering ways of dealing with out of signal email.  Both the DCC and ours are free services as best as I can tell, so don't know what the advertising would be for.  The traffic here is miniscule compared to a link I can put on other sites I run and I would guess probably similarly for Vernon.

Shelby Moore
http://AntiViotic.com

---

From: Shelby Moore <[email-masked]>
Date: Wed, 10 Sep 2003 06:20:43 +0800

I started this thread.

Let's please close this thread here on IETF and move it some where more appropriate.

I do not know exactly where to move it to (which list I will choose exactly because I want to research more to try to avoid running into Vernon, Valdis, Spencer, and others again which will be difficult since they seem to be every where).  What I may do is write the LD and make a working group some where and then apply to such a list and ask only those are really interested to follow.

Sometime in the future, I will make one more post on this list to say where all my replies will be going.

So if there are any more posts, I will be collecting them and making replies on them to another list in future.

Shelby Moore
http://AntiViotic.com

---

From: Dean Anderson <[email-masked]>
Date: Tue, 9 Sep 2003 18:42:17 -0400 (EDT)

On Wed, 10 Sep 2003, Shelby Moore wrote:

> However, I think the analysis of the concepts of information theory,
> channels, and models of spam is more fundamental to "internet
> engineering" than the original purpose of this thread and thus I see no
> reason why it would not be useful data here at IETF.

I tend to think that the more theoretical aspects of abuse-free protocols
aren't specific to spam, and are probably of more general interest.
However, discussion needs to be productive, as well as interesting.

> Before I respond to your continuance of your argument, I *respectfully*
> remind that I already refuted the whole line of criticism you are
> continuing in this post, when I rebutted your last post in this thread:

I did read your post. But it seems we have unbridgeable differences:

  > but to focus your attention on detection, rather than protocol
  > alteration.  It is impossible to alter the protocol in any way that
  > will force the spammer to identify themselves a-priori as a spammer.

  Disagree strongly.  First benefit is once you define spam == *BE
  (instead of UBE), then it is easier to model spam and do research on
  it,because you can model it at any node in the channel, not only at the
  receiver end point.  That was my whole point about "enforcers".

I think I've already covered this, but perhaps more clarity will be
helpful:

The theoretical definition of "spam" is simply "email that is unintended
by the the system operator, that is hidden from the system operator, and
violates the security policy of the system operator".  No more precision
is necessary to use the theoretical framework I am using.  We don't need
the exact intentions of the system operator, nor the security policy. We
just have to posit that these exist, and that they are violated by spam.

For example, altering the definition of spam doen't have any effect on the
impossibility of making a protocol that will automatically force the
spammer (uncooperatively) to distingish themselves from regular users
(self marking).  Self-marking could only be done with the spammers
cooperation, and there is no reason to think that todays abusers will
cooperate, since most of those abusers are already unconvicted felons and
have no commercial purposes.  Frequently, these abusers appear to use
viruses, which could be using the credentials of regular users. So, they
can do _everything_ those regular users can do.

Because you can't count on any quick cooperation before the recipients
mailbox, it doesn't seem to be helpful to think about modeling spam at
other nodes.

		--Dean

---

From: Bill Strahm <[email-masked]>
Date: Tue, 9 Sep 2003 18:28:47 -0400

Very nice.  I say to post an Internet Draft - you post a link to a simple 
archived e-mail.  The IETF process starts with an Internet Draft - without it
we are all just wasting time.  An internet draft is a concrete proposal that
can be discussed, archived, debated successfully, etc.

I challenge you to take your e-mail posting and turn it into an Internet Draft
with a legitimate security section (since you are solving a security problem)
then post a single message to this list showing the internet draft, and a
mailing list that people can join if they are interested in discussing it
further.  

From there it is easy to determine if there is enough interest in forming a BOF
in Minnesota ( or S. Korea in March ) to forward the work in a Working Group.

The problem you have run into is you haven't taken the first step, which is to
simply submit an Internet Draft to the Internet Drafts editor... very simple
process with no politics in the way.  From there you can pursue forming a 
Working Group (where you will get your first taste of politics).  Being that
you haven't taken the first step (publishing an Internet Draft) I am not sure
you "really" meet the charter (Ok, yes you do, but putting out a draft is SO
important - it should be the first step) and you have allowed the topic to grow
WAY beyond initial discussions (I am actually waiting for Harold to post one
of his famous Top n Talker lists soon).  The next step is to get a mailing
list somewhere and move the discussion off of this list onto that list

Bill

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 10 Sep 2003 01:15:18 +0200

> protocol.  That goal is impossible. Thus, we have abusable protocols.

Ok, not going to argue on that point (not necessarily agreeing, though).

> accept the premise that it impossible to create an abuse-free protocol.

My goal is to drastically lower unsullicited bulk mail. It's not the 
occasional message that bothers me, but the dozens each day that try to 
exploit human kind's lowest common denominator. I believe bulk and 
unsollicited can both be determined objectively, so there should be 
ample avenues to explore here.

> emphasis of definition, not loudness.)

Ok.

> from A to B.

Ok.

> exclusive list of what is permitted, but illustrates that spam isn't

Hm, is it? In most cases spamminess is pretty obvious, but I've both 
received and sent out complaints about spam which turned out to be 
fairly legitimate communication that just wasn't all that welcome any 
more. (But hard to tell when businesses change their (domain) name.) 
This stuff is hard to determine on a per-message basis.

The part that I care about is that spam is unwelcome in two ways: each 
individual spam message is unwanted by the recipient, and the generated 
system load is unwanted by the service provider.

> not permitted.  Various methods have been implemented to enforce this

Again, too hard to determine by just looking at a message somewhere in 
the middle of the path. Now if we define that a single user may only 
send 1000 messages a day, that's something we can work with.

> opposite output. (Seems that perpetual motion may have some analog 

Yes, this makes the whole thing somewhat harder...

> Speak for yourself.

You are the one who said "we".  :-)

I do think Shelby was on the right track with the idea of separating 
mailinglists from regular one-to-few email. (Not quite sold on the pop 
thing, though, that would be going 8 years back in time for me.) 
Mailinglists usually already have reasonable access controls and a 
single or very limited number of sources, so those shouldn't be a huge 
problem. With the sollicited bulk email out of the way, any bulk email 
that's left is automatically unsollicited (or a corner case that we can 
ignore at the price of some inconvenience) so this can be stopped or 
slowed down.

> AUTH has no effect on spam.

If I know who sent something I can filter so I don't have to receive 
any future filth from the same source. And if spammers are forced to 
use their real identity then finally they'll get at least some of what 
they have coming.

> If regular users can inject arbitrary

Disagree.

> protocols that require cooperation of the spammer aren't likely to

What we need is a system where the only action that has any result is a 
valid action. Then either spammers have to spew out their filth out in 
the open without being able to hide, and bear the consequences (I don't 
normally condone violence but for spammers I'll make an exception) or 
slither back under the rock they came from.

> computers as well as email containing the virus itself.

We have to fix this at some point, not just for spam, but also for 
DDOS. (Compared to which spam is a minor annoyance.)

> Has this ever happened in 8+ years?  No.  Will it ever happen? No

That's the great thing about authentication: I can blacklist someone's 
credentials on _my_ system so they don't get to send email to me or my 
customers anymore. No need to wait for lazy ISPs anymore.

> authentication.

No. A proxy service wouldn't normally have access to an email identity. 
On MacOS the user even has to specifically allow applications to use 
passwords, so unless a worm manages to get root access or take over an 
application that can already send email, it wouldn't have access to the 
email authentication data.

Also, you can force the outgoing mail through an ISP which can then do 
bulk detection.

> are somehow outsiders getting in.

No, that's not the point. The point is: fool me once, shame on you. 
Fool me twice, shame on me. I don't want the same bad guy to fool me 
twice.

> chemistry call for such things.

At the very least we get to make sure the blame isn't shifted to 
innocent bystanders.

> abused, that it should be removed.

Again, not the point. The point is that people think that the From: 
line indicates where the message comes from. This is a broken 
assumption. So remove the line and people will realize than anyone can 
put any name at the bottom of a message and there is no longer any 
half-baked trust of something that is fake 37.5% of the time. (Assuming 
50% spam and 75% forged From: which are both reasonably in line with 
what many people see.)

Also, "forge" implies some kind of effort to pull of the deception. But 
with email no effort of any kind is required: just say whatever you 
want and the other side takes your word for it. And then to think email 
has been used as evidence in court. Insane is the only word I have for 
it.

---

From: Vernon Schryver <[email-masked]>
Date: Mon, 8 Sep 2003 07:41:58 -0600 (MDT)

> b) there is no need for it to be 50% of all mail

That last sentiment is on my list.

There are several currently available, independent sets of mechanisms
that will keep more than 90% of all spam out of your mailbox with
fewer than 0.1% false positives.  If your mailbox receives on average
more than 1 spam/day and you care, then fire your current ISP and hire
one that offers reasonable spam defenses.  If you care to invest your
own time and effort maintaining filters or if you can tolerate more
than 0.1% false positives, your mailbox can be practically spam free.

Of course, if you fewer receive 0.1 messages/month, it's going to
be hard to keep the ratio of total mail to spam below 50%.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 11 Sep 2003 11:09:59 -0700

> fits the mission of no IETF group as I understand it (this for Harald).

Jefsey,

your suggested message format seems very similar to the 
"message/external-body" feature that has been part of MIME from the 
beginning.

as far as I can tell this suggestion is also already partially implemented 
by spammers, in the form of "the main spam is in a picture embedded in the 
HTML that is sent as email". I haven't seen any great benefits accruing 
from that in the form of making it more expensive to spam.

the restriction part turns weemail into a "gated community", where you have 
to have gatekeepers/converters between the weemail and the rest of the 
world, to convert mail into and out of "weemail" form. This violates the 
end-to-end principle of email (which is near the root of many of our 
problems with spam, but is also the main reason why the Internet mail 
system won out over X.400, PROFS and oh-so-many other mail systems).

when the gated community is larger than the rest of the world, this starts 
getting really interesting. But I don't see that there is a way to get 
there from here.....

                   Harald

---

From: jfcm <[email-masked]>
Date: Mon, 15 Sep 2003 13:28:46 +0200

At 20:09 11/09/03, Harald Tveit Alvestrand wrote:
>"message/external-body" feature that has been part of MIME from the beginning.

Thi sis true. But as far I understand there are only two options:
1. the address is to be to a mail server. [email-masked] and then access 
commands in the core of the text
2. or there are a limited set of formated entries in the header.

The weemail's need is for a free format extended header - for customized 
parameters.

>from that in the form of making it more expensive to spam.

Yes. This is why the external-body seems not to be a solution. There is no 
information which can be passed to the mail agent for its own 
decision/filtering. It is a centralized process?

>the restriction part turns weemail into a "gated community", where you 
>have to have gatekeepers/converters between the weemail and the rest of 
>the world, to convert mail into and out of "weemail" form. This violates 
>the end-to-end principle of email (which is near the root of many of our 

Yes and no. Yes if the filtering is made at a gateway. No if the filtering 
is made at the end. This is an option of the filters to be supported by 
MTA, OPES or by the agen user.

>getting really interesting. But I don't see that there is a way to get 

True for the Global Internet Community. Not necessarily for Private/Local 
Internet Communities.

This is aslo the old "dump net/smart host vs smart net/dump terminals" 
story. As community oriented we are more interested in "smart net/smart 
host". This is true, you may see/object to it both ways.

Nevertheless I am intrested in knowing about library suppliers and mail 
agent providers supporting message/external-body?

Thank you.
jfc

---

From: [email-masked]
Date: Mon, 15 Sep 2003 11:32:08 -0400

> The weemail's need is for a free format extended header - for customized 

If you can quantify the parameters and extensions well enough to put them
in an I-D, we'll take a look.   There's nothing that says we can't add stuff - we
added 'format=flowed' to text/plain, and there's been other things tweaked
over the past decade of MIME.

Keep in mind that except for the actual reference parameter, /external-body
carries around a bunch of other headers describing what you are going to fetch.
 It might be a good idea to think about it more generally, as additional info
intended to help me decide if I want to fetch a remote part is probably
remarkably similar to info that would help me decide whether to open a local
part (It's often the case that *opening* that .doc format file will involve
launching OpenOffice or a PDF viewer - on my laptop, this ends up being more
resource-intensive than downloading the file). 

> host". This is true, you may see/object to it both ways.

As a community *we* may be interested in both the net and host being "smart".
There are certain large groups with lots of money that wish that the average
host be barely smart enough to consume media only in the ways that the provider
intended.  Think "XBox/WevTV with DRM stuck on it" - there's a LOT of people
who want that to work.