Subject: [Asrg] Simple way to verify sender, track mail abusers

From: Dennis Gearon <[email-masked]>
Date: Thu, 25 Sep 2003 16:29:34 -0700

First of all, I hope this is a good place to send or propose or get 
comments on this idea.
Secondly, I welcome not only comments, but pointers to what is already 
being done in this task.
Third, also, any comments about the usefulness of identifying the sender 
would be welcome also.

<<background>>-------------------------------------------------------
I have looked at drafts and standards for verifying the user sending and 
email. I did not see any simple way to do it.

I looked at:
    http://www.imc.org/ids.html

and listings on it:
    http://www.imc.org/draft-ietf-smime-sender-auth
    http://www.imc.org/draft-church-dns-mail-sender
    http://www.imc.org/draft-fecyk-dsprotocol
    Some of the background knowledge, especially DNS related technology, 
is beyond me.

and:
    http://www.imc.org/rfcs.html#hosttohost

but found:
     no listings on it related to FROM verification.

<<idea>>--------------------------------------------------------------
Idea: allow mail servers to verify that:
    A/ Message is actually from a sender.
    B/ Body of message is the same as originally sent
    C/ Relay from identified, authorized users only, and using REAL 
FROM: address.
    D/ verify to receiving mail server the inverse, that authorized 
local client sent mail from server, or that message
        id, body, subjcect, to, and FROM: address came from a user on 
FROM: address system, no matter from
        where sent.

-------------------------------------------------------------------------------------------------------------
To send mail from authorized mail domain, mail client does following 
actions:
    Send mail as usual.

Server responds by:
    1/ Veriying password and username
    2/ Creates MESSAGE-ID
    3/ Selects 32 equidistant locations in the binary concatenation of 
FROM:, DATE:, SUBJECT:, and body of the email.
        prevents just appending spam to the end of a message, which 
could happen if mail is intercepted.
    4/ Harvests 2 bytes from each location
    5/ Creates an MD5 hash of ( username + DATE: + SUBJECT + MESSAGE-ID 
+ 64 harvested bytes )
        adds MD5 hash as header titled
            UDSMIDEqui32-Message-Hash:
    6/ Stores MESSAGE-ID (if it doesnt' already) with MD5 hash and username
    6/ Sends mail as usual

Receiving server does:
    1/ Contacts server in FROM: field.
    2/ If message contains header of 'From-Verified:', that header is 
removed.
    3/ Repeats steps 3 through 5 above.
    4/ Requests verification of sending server that FROM: user sent 
message with MESSAGE-ID and MD5 hash calculated.
        If MD5 hash does not match can refuse reciept,
        if mail is stored for user, whether verified or not, an 
additional header
        is inserted called From-Verified: which has values of:
            TRUE
            FALSE

-------------------------------------------------------------------------------------------------------------
To send mail from a different server than authorized mail domain, mail 
client does following actions:
    Send mail as usual.

Server responds by:
    1/ Selects 32 equidistant locations in the binary concatenation of 
FROM:, DATE:, SUBJECT:, and body of the email.
    2/ Harvests 2 bytes from each location
    3/ Creates Local Message ID
    4/ Contacts mail server in FROM: address.
    5/ Sends via an encrypted path:
        user_name,
        password,
        DATE:,
        SUBJECT
        64 harvested bytes
        Local Message ID
        and requests verification of user.
    6/ FROM: mail server validates user_name and password, creates:
        MESSAGE-ID,
        MD5 hash of:
            username
            DATE:
            SUBJECT
            MESSAGE-ID
            64 harvested bytes
        then stores
            MESSAGE-ID
            MD5 hash
            username.
            remote server name
    7/ FROM: mail server then sends via encrypted path:
        if username and password were matched:
            Received Local Message ID,
            created MESSAGE-ID
            MD5 hash
                -----OR-------
        or if username and password were NOT matched.
            Received Local Message ID
            FALSE keyword
    8/ Remote server assembles message as normal, substituting:
        received MESSAGE-ID for any MESSAGE-ID it would normally create
        adds MD5 hash as header titled
            UDSMID-Equi32-Message-Hash
    9/ sends mail as usual

Receiving server does:
    nothing different

-------------------------------------------------------------------------------------------------------------
additional notes.

If querying for username password cracking is a concern, remote sending 
can be authorized per user only once per minute or somethign like that, 
with maybe a ceiling per day, ceiling since last email download, and 
messages to user inserted into mail spool to inform them about status of 
verification requests.

The MD5 routine only executes over a fairly small data set and so 
consumes a small amount of CPU time

The messages between servers are very small, and so the encryption 
routines don't consume much CPU time, not does it add significantly to 
the internet traffic load.

Since the TO: field is NOT included in the hash, CC, BCC, and multple TO 
emailing is not affected; All messages sent at the same time

The logs for the server would be a little bigger, and would be better 
kept in a Database for each current month and one day off the tail 
flushed to a flat file daily.

Empty bodies are not a problem because the following headers
    FROM::
    SUBJECT:
    DATE:

These fields are 52 characters long WITHOUT EVEN ADDING address to 
FROM:, and if encoding header is added, the 64 bytes are

already exceeded without address. If necessary, message can be padded 
with /x00 bytes.

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Yakov Shafranovich <[email-masked]>
Date: Thu, 25 Sep 2003 19:43:18 -0400

Welcome to the group Dennis!

First of all, please follow the posting guidelines for this list (see 
http://www.irtf.org/asrg/asrg_mailing_list_information.htm). I changed 
the subject.

Second, please take a look at this presentation that outlines different 
approaches for email path verification:

http://www.elan.net/~william/asrg-emailpathverification-presentation.pdf

Third, take a look at the technical considerations and requirements 
documents, specifically in regards to anonymity. Sender verification 
proposals tend to conflict with the idea of anonymity and that must be 
take into account. The documents are available at:

http://www.ietf.org/internet-drafts/draft-crocker-spam-techconsider-02.txt
http://www.infobro.com/anon-FTP/infoSource/IRTF/ASRG/draft-irtf-asrg-requirements-xx-05.txt

Fourth, take a look at the CRI proposal:

http://www.ietf.org/internet-drafts/draft-irtf-asrg-cri-00.txt

Dennis Gearon wrote:
>    C/ Relay from identified, authorized users only, and using REAL FROM: 
> address.
>    D/ verify to receiving mail server the inverse, that authorized local 
> client sent mail from server, or that message
>        prevents just appending spam to the end of a message, which could 
> happen if mail is intercepted.
>    5/ Creates an MD5 hash of ( username + DATE: + SUBJECT + MESSAGE-ID + 
> 64 harvested bytes )

---

From: Dennis Gearon <[email-masked]>
Date: Thu, 25 Sep 2003 17:32:22 -0700

> http://www.ietf.org/internet-drafts/draft-irtf-asrg-cri-00.txt

Actually, my idea, I believe, takes care of most of the problems with 
the Level 2 CRI system. I will think about it for awhile.

---

From: david nicol <[email-masked]>
Date: 28 Sep 2003 18:08:16 -0500

> the Level 2 CRI system. I will think about it for awhile.

When is CRI 01 due?  Who is maintaining it (I nominate Eric Dean)?
It has been pointed out that body hashing would make CRI level-2
work, and this was AIUI generally agreed-to.  Yet a revised CRI
document has not yet appeared.

-- 
David Nicol /  kernel 2.6.0 is pretty whippy

---

From: "Eric Dean" <[email-masked]>
Date: Sun, 28 Sep 2003 22:55:14 -0400

Considering I haven't received but a comment or two, sure, I'll maintain
the draft.

With regards to body hashing or anything else, no one is preventing such
a method.  It's an option available to individual implementers of CR
systems.  I am merely proposing a method that they interoperate.

I am not proposing that CR is a solution for spam.  In fact, I have
evidence to the contrary...nevertheless, they exist and are used in many
places throughout the Internet.  IMHO it may be an interesting idea if
they automatically interoperate rather than require user interaction.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
david
> nicol
> Sent: Sunday, September 28, 2003 7:08 PM
> To: Dennis Gearon
> Cc: [email-masked]
> Subject: Re: 6. Proposals - Sender Verification (was Re: [Asrg] Simple
> wayto verify sender, track mail abusers)
>

---

From: Dennis Gearon <[email-masked]>
Date: Sun, 28 Sep 2003 21:14:26 -0700

>document has not yet appeared.
I suggested both body and specific headers be hashed. And only 32 bytes 
from the body to save processor time. Someone gave some good, valid 
reasons why the CRI/hash idea has problems, but I think they would not 
be too hard to overcome. Mostly, it was:

    A/ The need to keep records of what is sent.
    B/ The additional changes to SMTP required
        (Anyone really believe that SMTP will forever be extendible and 
not have to be replaced someday?)
    C/ Messages can't be stopped before they leave, only when they arrive.

Well, I am too tired and not well enough versed to coment on A and B. 
However, regarding CI think that the whole SPAM problem has to be solved 
in a two part solution anyway:
    A/ Develop a way to know who is sending what, and receivers can 
decide to accept or not mail that is not certified to be from who it 
says it is from.
    B/ Implement blocking on the send or receive sides, using lists, or 
authority, or validation, etc.

My proposal, and CRI/hash in general only addresses A. I leave B to more 
knowledgable people.

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 29 Sep 2003 01:43:17 -0400

> document has not yet appeared.

Hi David,

The point person on the CRI proposal is Eric Dean. HOWEVER, there are 
areas where the CRI proposals crosses over into the consent framework, 
and it will hopefully evolve into a general consent token exchange 
protocol within that framework. The CRI proposal allows for other token 
types except "plain" as an extension to the protocol, and hashes can be 
used for that. If you would like to participate in the CRI proposal, 
contact Eric and myself off-list.

Yakov

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 29 Sep 2003 02:01:49 -0400

For those who do not like the entire idea of C/R, it might be useful if 
someone volunteers to write up an evaluation of C/R in general. 
Something on C/R was mentioned in the technical considerations document 
which can be used in conjunction with the requirements document to do an 
evaluation. This way we can see a point by point break down of cons and 
pros of C/R. William Leibzon also has a presentation relevant to this. 
The documents are available at:

There is also some BCPs that can be written up for C/R systems. Some of 
that work has been done in the CRI proposal and Brad Templenton also has 
a BCP list. These are available at:

http://www.ietf.org/internet-drafts/draft-irtf-asrg-cri-00.txt
http://www.templetons.com/brad/spam/challengeresponse.html

Eric Dean wrote:
>>>Actually, my idea, I believe, takes care of most of the problems
> 
> with
>

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 29 Sep 2003 02:10:44 -0400

> decide to accept or not mail that is not certified to be from who it 

Also take a look at:

> authority, or validation, etc.

Take a look at:

http://www.solidmatrix.com/research/asrg/asrg-consent-framework.html

---

From: "Eric Dean" <[email-masked]>
Date: Mon, 29 Sep 2003 09:33:16 -0400

While I believe that there is a considerable amount of research,
brainstorming, and intuition that could go into such a document.  I
would be very interested in seeing some statistics from CR people our
there.

For example,

1) What percentage (and volume) of challenges are responded?
2) What percentage of challenges bounce?
3) What percentage of whitelisted senders are from CR vs. manual (or
from outgoing email, Contacts-import....)
4) What percentage of spam still gets through..and how/why?
5) What percentage of users (and volume) use the CR system for
anti-spam?  How long to acclimate?

Then there are qualitative issues of lessons-learned, perception,
effectiveness....

> From: [email-masked]
[mailto:[email-masked]]
> On Behalf Of Yakov Shafranovich
> Sent: Monday, September 29, 2003 2:02 AM
> To: Eric Dean
> Cc: 'david nicol'; 'Dennis Gearon'; [email-masked]; Brad Knowles
http://www.ietf.org/internet-drafts/draft-crocker-spam-techconsider-02.t
xt