Subject: [Asrg] Consent proposal

From: [email-masked]
Date: Mon, 30 Jun 2003 11:40:28 -0500

> ...This can be considered a consent-based concept, as it can be a parameter at 
the ISP end for each mail user. It adds quite a bit of overhead to the mail
server as it requires two additional messages for each one sent.  I see it
being part of the MTA process.  I send a message to someone.  The MTA
replies to the message with a "proof of sender request, " basically a
specially formatted email message that goes back to the server that sent it
based on the return address.  

The idea sounds promising other than that initial, 
perhaps-not-quite-fatally-flawed premise.  You've fallen into a big trap that 
confuses many folks... folks who don't realize that SMTP mail servers and POP3 
mail servers represent in fact two quite different entities (despite the fact 
that SMTP servers at some point talk to POP3 servers).  Specifically, at the 
sender's send, the SMTP and POP3 mail servers are not only not (usually) the 
same program, they are *quite* often not on the same machine (and moreover, may 
not even be under the ownership of the same company!).

Thus, if a recipient MTA sends a message back "based on the return address" it 
will go to the sender's POP3 server, which is VERY likely *not* going to be "the 
server that sent" the original outgoing SMTP message.

For one example, let's take a look at the (rather convoluted) path that my 
E-mail messages go through:

1)  I'm composing my messages here within my home LAN, behind a NAT router and 
cable modem.  My return address is my own domain, terabites.com.  My E-mail 
leaves my mail client and is passed in turn through my EMWAC SMTP server running 
on a NT 4.0 server box.  That passes it upstream through my WinProxy proxy 
server (running on a Win2K server box) which then goes out through my firewall, 
through the NAT router to the cable modem, and goes (obligatorially) to AT&T 
broadband (becoming Comcast's) outgoing SMTP server.  Note that many ISPs 
require that ALL outgoing mail from their users go through their SMTP servers, 
and do not allow downstream nodes to connect to off-net SMTP servers.

2)  AT&T's SMTP server delivers the message to the recipient's indicated domain, 
which might (or might not!) be the actual recipient's POP3 server.  Let's 
consider my own case, though, where the mail will be delivered to my personal 
domain provider (terabites.com).  NOTE THAT IT HERE IS SERVING AS RECIPIENT, NOT 
AS A SENDER!

3)  My domain provider's MTA forwards the incoming mail as requested to 
gordonepeterson2 at attbi.com.

4)  AT&T Broadband's (recently acquired by Comcast) mail server forwards my mail 
(starting today, in fact) to Comcast's new mail server at mail.comcast.net.  
This is the POP3 mail queue where I will (starting today) go to fetch my 
incoming messages.

Okay, so now this server at mail.comcast.net sends a request to 
[email-masked]'s MTA requesting confirmation that it originated the message 
in question.  It clearly did not... it was an intermediate point.

So who, anyhow, was actually the "originating" mail server, and how would one 
get back to it?

Let's say that Domain Direct (my domain provider) just forwards the 
'confirmation request' on, like it forwards the rest of my mail.  So it forwards 
the mail on through to gordonepeterson2 at attbi.com, which dutifully (starting 
today) forwards it on through to gordonepeterson2 at comcast.net.  Comcast's 
mail server, of course, has never seen the message I originally sent, so it 
can't confirm that it sent the original message.  

Now in fact, attbi's server (NOT mail.attbi.com, but smtp.attbi.com) IS the mail 
server that (as far as the Net at large is concerned) "originated" the message, 
although it probably wouldn't have any way to notice that (and certainly not 
based on my [email-masked] E-mail address).  And then there are my proxy 
server and the REAL originating SMTP mail server at nts1.terabites.com, although 
it's behind a NAT router so nobody outside my local LAN can even get to that 
(TRUE originating) SMTP server at its non-routable IP address.

Let's take a look at another interesting variant on this, and that's Yahoogroups 
(specfically, but other mailing lists have similar issues):

So anyhow, the first step above is the same, then my post goes to (say) 
[email-masked] which forwards it on to the recipients of that 
mailing list.  Let's take the example for now of where the messages are 
delivered individually to list subscribers.

The mail has the "from" and "replyto" addresses of either (at the list 
moderator's option) [email-masked] or [email-masked].

Sending a confirmation request to [email-masked] as the originator takes us 
back through the earlier situation, with the difficulties now compounded by the 
fact that (1) the message subject now has a (by default) [mailinglistname] 
prefixed to it, and that (2) NONE of the mail servers in the path we followed 
together above shows the message I originally sent going to 
[email-masked].  The "originator" in that case is actually of course the 
Yahoogroups (or other mailing list) server, although it's not guaranteed that 
any of the "return address" items in the group will point back to it.  (Mailing 
list owners can usually specify whether they want replies to go by default to 
the originators of the message in question, versus to the list).

Now, one COULD imagine a scheme whereby a sender's outgoing mail could contain 
some kind of new header line which specified that a (and which) mail server was 
prepared to take responsibility for sending the message in question... but if 
it's not "the mail server" (whatever THAT actually means) corresponding to the 
sender's E-mail (presumably return) address, then it's not clear that you've 
gained a whole lot... since a spammer could say "here, use THIS server for 
confirmations!" and point at a bogus "spammer server" which simply confirms ALL 
requests in the affirmative. 

But even disregarding that problem, a more basic issue with this whole avenue of 
thinking is that it has to be handled by basically the whole Net before it can 
be useful... it's not a scheme (as mine is) which can be implemented 
incrementally, starting with as little as one single POP3 server.

> The sender's MTA responds with a "confirmation
request"  if it did send the message and the message is now available for
download based on moving it from the pending directory to a user directory.
If there is no response within a certain period the message is discarded.
It's a similar concept that I use when I get a telemarketer who I may want
to pursue what they are selling.  I ask for their phone number and call then
back. Granted I don't do this very often as I usually tell them to take me
off their call list and hang up.  If this has been mentioned I apologize as
the volume of messages is quite large on this list.

> In terms of Yakovs' consent based system:  I like the idea of an ISP form
that allows me to check boxes to deny HTML or non-signed messages.  

I think it has to be more than that, because I'm willing to (grudgingly) accept 
HTML from a few senders who (1) I feel are justified in using HTML, and (2) I 
trust to not abuse the power it gives them.  What I think is CRUCIAL is to allow 
me to specify the addressee/sender address pair for which I give permission to 
send me HTML-burdened E-mails (and/or attachments and/or base64 encoded bodies 
or whatever).  

Here again there is the Yahoogroups mailing lists issue.  

I have all of my Yahoogroups mailing lists set to send me only plain ASCII text, 
so those should get to me regardless of whether I list those groups as "trusted 
senders" or not.  Note that even if the Yahoogroups list is set to leave the 
mail with the original real sender's E-mail as the From: address, since 
Yahoogroups is sending only plain ASCII text, it should come through fine.

> And probably enter a few basic filters like deny any message that mentions
Viagra.  

One problem there is that there are so many ways of obscuring the content.  For 
instance, spammers commonly will use V I A G R A or V1AGRA or VIA<foo>GRA or 
some such to try to defeat content scanners.  So filters need to end up being a 
lot more than just "basic" if they're going to be effective.  A simple grep-type 
search is nowhere near adequate.  (I *will* claim however that messages which by 
default are not allowed to contain HTML, attachments, or printable or other 
encoding deny to spammers a large percentage of the potential types of deceits 
and deceptions they can use to defeat content scanners.  Hence, I think that 
preventing untrusted senders from sending HTML-burdened messages by default is 
one MAJOR step that we NEED to employ to stem the crushing tidal wave of spam!). 
 My personal content filtering here is based on SNOBOL4/SPITBOL, whose pattern 
matching is far more powerful (AND faster!) than the braindead regex-based 
pattern matching offered in Perl, which is what most (other!) people have 
attempted to use for such content filters so far.

Another fairly obvious problem is that the message YOU just sent (which I'm 
responding to) would be filtered out by a filter triggered by the word "Viagra" 
in the E-mail message (and my just RE-mentioning it would make THIS one even 
more likely, perhaps, to be t-canned).  Those are issues of content filters in 
general, but regardless of the technology these are hugely simplified if they 
don't have to deal with HTML, scripting, and attachments.

> Also you could determine what lists to use to deny or accept
messages.  I prefer not to filter messages on my machine, but I don't see
the typical ISP processing all these rules and filters without a substantial
increase in cost of doing business. 

That's true, but perhaps not to the degree you'd indicate.  Remember that there 
are costs to passing along the tidal wave of spam, too... it might be that it 
costs less to truncate it earlier.  Processing power really is pretty cheap (and 
getting cheaper all the time) so adding even quite a bit of power for filtering 
doesn't have to really be very expensive (especially compared to the cost of 
bandwidth to receive and send on the messages to begin with).

> So I think having something at both ends makes sense in the short term.  

Certainly the really FINE-grained filtering should probably be done at the 
recipient end (and giving them the opportunity for "one last ditch effort" to 
filter out some of the residual spam which evaded the various filters higher up 
the chain.

> The long term it should be an issue for the MTA to manage based on some 
parameters that the user or ISP set.

That would be NICE but in the end, I think that final control of each user's 
inbox belongs with the user.

> I like the idea of a white list that is supported by some industry entity
like the DMA.  

I don't necessarily think that they're going to like a technique that's as 
restrictive as the users probably want.  

> But I also like the way I can put my phone number in the
DMA's database and all those marketers who subscribe to their service won't
call me.

If enough of the public do that, the DMA's list will incorporate most of the 
phone numbers in the USA and marketers simply won't join or subscribe to DMA's 
service.  That's why such lists must be maintained by the government instead, 
and then we just all hope and pray that big-political-contributor companies 
don't manage to subvert the will of the people (more than they already have).

:-((

> The bottom line is that consent should be easy for the general user to deal
with and understand.

On THAT part we're definitely on the same page.

How about something like:

mail to [email-masked]
	default
		allow plain ASCII
	from [email-masked]
		allow HTML only safe 
		allow attachments only (JPG,DOC)
	from [email-masked]
		allow encoding including (base64,printable)
	from [email-masked]
		allow attachments including (XLS)
	from [email-masked]
		allow HTML including (scripting,Javascript)
		allow attachments including executable
	from md5hash([email-masked])
		allow attachments including (JPG,ZIP,PGP)
		allow encoding including (base64)
	from [email-masked]
		allow attachments only (JPG,DOC)
	from [email-masked]
		allow HTML
	from [email-masked]
		allow attachments including executable
	from [email-masked]
		allow attachments including (TTF,GIF,DOC,PPT)
	from [email-masked]
		allow attachments
		allow HTML
mail to [email-masked]
	from [email-masked]
		allow attachments only (JPG,GIF)
mail to [email-masked]
	from [email-masked]
		allow attachments only (ZIP)
	from [email-masked]
		allow attachments only (XML,ZIP)

Initially, this permissions list could be simply edited, although longer-term it 
probably should be manipulated by incremental add/update through some kind of 
menu system, Web page, or maybe E-mailed commands.  A good implementation would 
probably maintain at least two copies of the list, one a local copy on the 
user's own machine (just for a safeguard backup copy) and the other at the 
user's ISP or domain provider (wherever the filtering is going to be done).

If one wanted to use hashing or something (I've hinted at that ability above) 
there are a variety of things that might be implemented there (again, we don't 
need worldwide consensus for any of my proposals, since they're basically 
single-ended and only involve the recipient user and their ISP or domain 
provider).  Presumably hashing or encryption would be done, if done at all, in 
whatever way would provide adequate security and not revealing secret 
information to those who shouldn't have it (and don't have it anyhow).

Gordon Peterson                  http://personal.terabites.com/
1977-2002  Twenty-fifth anniversary year of Local Area Networking!
Support the Anti-SPAM Amendment!  Join at http://www.cauce.org/
12/19/98: Partisan Republicans scornfully ignore the voters they "represent".
12/09/00: the date the Republican Party took down democracy in America.

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 30 Jun 2003 22:31:33 -0400

At 11:40 AM 6/30/2003 -0500, [email-masked] wrote:

>[..]

This is also a concern with C/R since it relies on a similar approach. 
Currently I believe C/R companies such as MailBlocks are web based and thus 
all of their servers are together. Implementing this approach and C/R, 
would cost users in flexibility to be able to work with multiple servers. 
This problem might be alleviated slightly by writing a protocol for these 
servers to communicate with each other, but then all kinds of other 
problems pop up (mainly resistance to change).

>incrementally, starting with as little as one single POP3 server.

Unless this POP3 server can communicate with the SMTP server. If a 
communication protocol exists, then this approach can start of with pairs 
of SMTP/POP3 servers. Another approach would be to do the verification in 
the MUA.

>[..]

Yes this has been mentioned prior.

>[..]

Lots of good ideas here.

Yakov