Subject: RE: [Asrg] DCC and IP checksums

From: "Sauer, Damon" <[email-masked]>
Date: Mon, 10 Mar 2003 09:25:14 -0600

>Hadmut Danisch wrote:
>> On Fri, Mar 07, 2003 at 06:54:08PM -0500, David F. Skoll wrote:
>> 
>>>I use the following algorithm in CanIt.  It is by no means perfect,
>>>but it's pretty good:

>>>  - Skip a line starting "Dear "

>>>- Do a SHA1 hash on what's left.

>> This won't work. 

>It works for certain classes of spammer. It can be made significantly 
>better (eg: stripping HTML tags and comments, stripping 
>non-alphanumerics, not only collapsing whitespace, but _removing_ it), 
>but it's still trivially bypassable.
<snip>

 I actually wrote a very simple perl script that works perfectly in the way
that you want the chksum to work.
The problem was that it classified every newsletter, breaking news alert,
and weather report as spam.
Duplicity checks don't work because there is too much valid duplicity, not
because it cannot be accomplished technically.

Regards,
Damon 

*****
"The information transmitted is intended only for the person or entity to
which it is addressed and may contain confidential, proprietary, and/or
privileged material.  Any review, retransmission, dissemination or other use
of, or taking of any action in reliance upon, this information by persons or
entities other than the intended recipient is prohibited.  If you received
this in error, please contact the sender and delete the material from all
computers."
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Hadmut Danisch <[email-masked]>
Date: Mon, 10 Mar 2003 16:44:12 +0100

On Mon, Mar 10, 2003 at 10:18:55AM -0500, Sauer, Damon wrote:
> I have actually come up with a way that this works perfectly. 
> It does not matter what the password is or the dear * line says.

In imitation of the coding theory's "hamming distance":

What's the maximum "spamming distance" of to messages resulting 
in the same hash value? 

In other words: How much can messages differ to still get the same
hash and what's the minimum difference that guarantees a different
hash?

Hadmut

---

From: "Sauer, Damon" <[email-masked]>
Date: Mon, 10 Mar 2003 12:02:08 -0500

I revealed nothing ;-) It does not matter one bit if a spammer knew exactly
what lines I was looking at, that was part of the beauty of the process.
 Regardless, it did not help my spam situation. I was just stating that
chksums do work, however, they are checking for duplicity and duplicity does
not equate to spam in all cases. Blocking based on duplicity (lets say over
100 of the same message) would block very valid and valuable email.

Regards,
Damon

-----Original Message-----
From: Hadmut Danisch [mailto:[email-masked]]
Sent: Monday, March 10, 2003 11:16 AM
To: Sauer, Damon
Cc: [email-masked]
Subject: Re: [Asrg] DCC and IP checksums

On Mon, Mar 10, 2003 at 11:08:15AM -0500, Sauer, Damon wrote:
> I had the code up and working for about a month. I found that doing an MD5
> of the hash result of 5 lines in the middle of the message worked
> just fine.

There's exactly one reason why this works: 

It is your very private and not publicly known method.

As soon as spammers get aware of this (and they will since you 
just revealed it on this list), your method won't work anymore,
since it is easy to circumvent once the spammer knows how it works. 

Such methods violate one of the most important rules:
Don't do "security by obscurity".

We need to develop methods which can be used by everyone on the world, 
thus need to remain effective when the spammers learned all details
of the method. Your MD5 method fails to do so.

BTW: How do you handle attachments? 

Hadmut

*****