Subject: Re: [Asrg] A possible way to a solution?

From: [email-masked]
Date: Fri, 21 Mar 2003 02:00:00 -0800 (PST)

Again I strongly advice you to view my slides and compare difference between 
"simple" and "complex" mail path and take a look at real-life example of
what happens to email when you send it to asrg mail list, you'll see how 
headers change and how you can not rely on either MAIL FROM or "From:" header
as far as what your receiving mail server sees in comparision to remote 
mail server. What you and many others see as solution are based on 
misunderstanding on how email travels across the internet - way too many 
only see it as "simple" mail path. 

Considering that you're not the first person who proposed this solution
(I'v been on way too many lists...) I'll probably add this to my slides as
far as failed verification ideas, so we don't get asked about it again!  
Also very similar thing to yours has been proposed (but it does include 
callback): http://www.ietf.org/internet-drafts/draft-nunn-ssmtp-00.txt
I'll leave it as an exercise for you to check why this does not work, but 
if you do not I actually have completed slide on this which did not 
make it into final draft and as I promised to bring up callback, I'll 
release those slides when I get a change to update this presentation.
(I'm considering converting it into html as work-paper describing 
different approaches and comparisons - slides were obviously something for 
live show I did not get a chance to do; if I convert it, it'll be some 
time before you see update as I'll need to be lot more descriptive as 
would be in research paper).

On Fri, 21 Mar 2003, Olger Diekstra wrote:

> I didn't mean to use DNS for authentication.
> I'll illustrate the method I had in mind for this. If my receiving mailserver gets a message from a 
> remote mailserver, it'll use the emailheader which contains the FROM field (basically, I think the 
> FROM field should contain the senders address, like there's ultimately one TO field that really 
> matters, there should also be one FROM field that contains the actual senders address. That's the 
> one I'm gonna check.). From that field I take the domainname, look it up in DNS, get the 
> mailserver's IP address, and then contact that mailserver. Using a new protocol my mailserver 
> communicates with the sending mailserver, to find out whether or not it is actually trying to send 
> an email to my mailserver.
> If the domain in the FROM emailaddress nonexistence, the email will get bounced. The sender will 
> get notified by means of an email. The sender can then take appropriate action, and eventually 
> resend his/her email. With a correct emailaddress.
> DNS is used as it was intended, not for authentication, not with rDNS.
> 
> There's no point in trying to find a solution that will totally eliminate spam. It might work for a 
> while, but anything is ultimately breakable. In my opinion, its better to make it very hard to 
> spam, and/or give a spammer a lot of work to sending his/her email.
> Just look at the all the copy protection schema's that are around. They make it harder to copy a CD 
> or DVD for a while, not impossible. 
> 
> Olger.
> 
> >Once again this is type of verification based on DNS, I described
> >majority of this approaches in my presentation and in the beginning
> >described what these approaches would need to deal with and the
> >solution described here is pretty much what I called complex mail
> >verficiation. Again ANYBODY who is considering doing any kind DNS
> >based mail sender verification any kind of solution like has been
> >discussed at "Domain Authorized SMTP Mail" thread and RMX and number
> >of other threads (and there are many many people like this on the
> >list), I strongly recommend to read my presentation at:
> >
> >http://www.elan.net/~william/asrg-emailpathverification-
> >presentation.pdf
> >
> >(Note that right now it looks like I will not be presenting this on
> >actual meeting as chair has chosen different presentations for it)
> >
> >I also do think that being able to track spammer to his real
> >location will go 1/2 way in stopping them, the rest would fall into
> >being able to contstruct good black/whitelists and doing legal and
> >other actions to close spammer business.
> >
> >On Thu, 20 Mar 2003, Olger Diekstra wrote:
> >
> >>Hi all,
> >>
> >>I've been monitoring the list for the past few days now. Hearing
> >>(well, reading) everyone with all the different types of solutions
> >>and arguments brings me to the following concept. Although its far
> >>from covering everything, there are still some lose ends, I think
> >>its a good starting point.
> >>
> >>Although spam should be (and in most countries is) illegal, that's
> >>not going to stop it. As long as there are countries that are more
> >>flexible to spammers, it will be hard to get them before court.
> >>Which ultimately leads to conclude that the law is not going to be
> >>able to solve this. So any discussion whether it should be more
> >>illegal or not, heavier punishments or cheaper ways to bring a
> >>spammer to justice will not make spam go away. In other words, a
> >>futile discussion.
> >>Since justice needs a person or organization to stand trial, what
> >>we need to do first is find out who is spamming in a legal way.
> >>Then Justice can take place much more easily.
> >>
> >>Content filtering is (as has been pointed out before) ultimately an
> >>enduser solution. As email can be encrypted (although encrypted
> >>spam is hardly very likely) it becomes difficult for an automated
> >>process to be checked. Besides, the same content can sometimes be
> >>wanted and sometimes be spam (as has also been described in emails
> >>from others on the list). If I like to write dirty to my
> >>girlfriend, that is not spam. However, if it is to thousands of
> >>users around the world that I do not know, that would make it spam
> >>for most, although there might still be a few who would want that
> >>kind of mail...
> >>
> >>Systems that rely on remote systems setting tags, or having
> >>certificates are also not very reliable in the end. They can be
> >>forged. If a spammer has its own mailserver, it can forge any tag
> >>that is invented to prevent spam.
> >>
> >>Seperation of spam and legitimate mail must be close to 100%.
> >>Something like 99.99999%. If on a billion mails only one gets lost,
> >>that's acceptable to me.
> >>Furthermore, the traffic spam consumes must be reduced to its
> >>absoluut minimum. Determining if an email is spam can only be done
> >>at the receiving side on the server or on the client. A spammer can
> >>be a legitimate sender at one time, and a spammer at another.
> >>
> >>Which ultimately leads me to the conclusion that the receiving
> >>mailserver should be the first place to start. This is the place
> >>where the spammer has no control over.
> >>If he knocks on my door and doesn't know the password, I can simply
> >>keep the door locked.
> >>Whatever authentication or certification my mailserver wants to
> >>receive an email, the spammer will have to comply to. (As long as
> >>my security on the server is sound of course...But even then, if
> >>security is low, the spammer will have to be a hacker as well,
> >>totally different skills, and much more timeconsuming).
> >>
> >>So the question is probably, how do we do that?
> >>Firstly, I want my mailserver to garantee the sender. That will
> >>give me much more control. The server could do this as follows:
> >>whenever a remote mailserver sends an email to a receiving
> >>mailserver, the receiving mailserver should lookup the domain of
> >>which the sender claims the email is from, contact the mailserver
> >>of that domain, and check it is actually trying to send the email.
> >>This will garantee the source to my mailserver, giving me full
> >>control over the check. It will be very difficult for the spammer
> >>to forge this.
> >>Now a problem rises for users that send from a mailserver that's is
> >>not part of the source domain. Like mobile users sending mail from
> >>their hotelroom, home users sending with an office emailaddress,
> >>etc. This can be solved by informing the mailserver of the domain
> >>in the FROM field that a message is being send. Basically, you
> >>could send the header of the email which will provide enough
> >>information. For this to happen reliably, the user would have to
> >>login to the mailserver. In the end, if the mailserver of the
> >>domain in the FROM field is not notified, the sender will not be
> >>able to get the email to me.
> >>
> >>So now that I know the source,  I could setup a list of users on
> >>the mailserver that are save. Whenever the mailserver receives an
> >>email for me, it'll check against the list of known addresses. If
> >>the sending party is in my list, the email can go right to my
> >>mailbox. If not, additional action is needed (and has to be
> >>defined). Very safe, and without too much need for server
> >>resources.
> >>I can still send email from anywhere in the world to anyone. Thus,
> >>I can create a whitelist of folks whom I trust. Of course, I would
> >>need to setup all my emailboxes with the necessary filters.
> >>
> >>This mechanisme ensures that an email can be traced back to the
> >>sending domain, it has to have a valid sending emailaddress of
> >>which the domainname is registered in DNS and the authorized
> >>mailserver of that domain must have send the email.
> >>If more mailservers exists within the sending domain, it is likely
> >>that the receiving mailserver connects to a different mailserver
> >>from the sending domain. So there also has to be a mechanisme for
> >>the sending mailservers to communicate amongst each other.
> >>Basically, it should do a sort of broadcast to is fellow
> >>mailservers of the domain, and ask if anyone is sending the email.
> >>
> >>Since we now know who is sending the spam (forging emailaddresses
> >>just became much more difficult) we can put up a blacklist with
> >>known spammers. An email server should be able to either mirror it
> >>in case the blacklist goes down (due to a DoS attack of some sort)
> >>or use it realtime. Every user should have an option somewhere, in
> >>its client, thru a website of the ISP, whatever, to select a
> >>particular blacklist and tell the emailserver to use that list or
> >>lists for the users mailbox. The email server will first use the
> >>users whitelist, and then the blacklist.
> >>If a user considers an email spam, it should be able to reply the
> >>email and sent it out again, using a special tag in the subject
> >>stating SPAM or something. The email server will pick this up and
> >>instead of sending the reply to the spammer (though he might get
> >>perhaps a copy?) it will sent it to the blacklist.
> >>If the sending address gets, say, twenty emails from different
> >>persons stating the sender is a spammer, it could be added to the
> >>list.
> >>
> >>Spammers who use a temporary hotmail account might still form a
> >>problem, but they will not be able to send email using that account
> >>if they do not notify the hotmail emailserver. Webmail accounts
> >>usually can only be used from a webpage which sends the email using
> >>an unknown (to the sender) mailserver. I have a feeling spammers
> >>are not going to use a webpage to construct thousands of email
> >>spam....
> >>
> >>Remember, this is just a thought (though a long one...) of how it
> >>could be done. The above described method can be slowly implemented
> >>in the existing systems.
> >>If users also authenticate to SMTP servers, the ISP or
> >>administrator can also make sure no unauthorised person gets to use
> >>the mailserver. In the end, it will make spamming a whole lot
> >>harder.
> >>
> >>Happy thinking!
> >>
> >>Olger Diekstra.
> >>
> >>
> >>_______________________________________________ Asrg mailing list
> >>[email-masked] https://www1.ietf.org/mailman/listinfo/asrg
> >>

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: David Walker <[email-masked]>
Date: Fri, 21 Mar 2003 08:26:42 -0600

There is a reason these ideas keep getting proposed.  The idea is solid. 
(Different descriptions do overlook certain items but those items can all be 
solved.)  Implemented properly it is no more of a load than a blacklist and a 
lot less subject to the whims of a madman than the blacklists that exist now.

As far as "simple" vs "complex", for all the complexities you may add every 
mail must travel the simple path when traveling between domains.

Take mailing list mail, since that seems to be one of the more complex ideas.  
You send a message from your workstation which travels among any number of 
complex patterns until it must leave your domain and travel to the domain of 
the mailing list post address.  It must traverse the "simple" path to get 
there.  Then the mailing list software processes it and sends it out to the 
list.  Each of those deliveries must also traverse the "simple" path.

Currently most mailing list servers rely on the MAIL FROM to 
pseudo-authenticate mailing list members.  With a working dns scheme they 
could at least ensure that the sender was using an outgoing mail server 
approved by their domain administrator.

The MAIL FROM does not represent the entire path of the message.  It only 
really represents a bounce path back to the server that is currently 
accepting responsibility for that message.  In essence, in our mailing list 
example there are 2 separate messages.  
Simple Path #1: The poster is claiming "I am [email-masked] and I have a 
message for [email-masked]"

Simple Path #2:  The mail server is claiming "I am [email-masked] and I 
have a message for [email-masked]"

These should not be viewed as a single message.  They are 2 distinct messages.

On Friday 21 March 2003 04:00 am, [email-masked] wrote:
> Again I strongly advice you to view my slides and compare difference
> between "simple" and "complex" mail path and take a look at real-life
> example of what happens to email when you send it to asrg mail list, you'll
> see how headers change and how you can not rely on either MAIL FROM or
> "From:" header as far as what your receiving mail server sees in
> comparision to remote mail server. What you and many others see as solution
> are based on misunderstanding on how email travels across the internet -
> way too many only see it as "simple" mail path.
>

---

From: Dave Crocker <[email-masked]>
Date: Fri, 21 Mar 2003 09:13:04 -0800

David,

Friday, March 21, 2003, 6:26:42 AM, you wrote:
DW> There is a reason these ideas keep getting proposed.  The idea is solid.

Many ideas that are otherwise wonderful prove unworkable when they
must be applied to a working system. Many ideas that are otherwise
wonderful prove unworkable when they must be applied to a system with
a hundred million users.

Too often, ideas are evaluated on their basic appeal, rather than
their serious feasibility in terms of deployment and use within an
existing, large-scale service.

d/
--
 Dave Crocker <mailto:[email-masked]>
 Brandenburg InternetWorking <http://www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>, <fax:+1.866.358.5301>

---

From: David Walker <[email-masked]>
Date: Fri, 21 Mar 2003 11:40:20 -0600

You are absolutely correct.  In my opinion these schemes are being rejected 
because they lack some basic appeal, without regard to their feasibility in 
terms of deployment and use within an existing large-scale service.