Subject: RE: [Asrg] 2.a.1 Analysis of Actual Spam Data - Titan Key reduces spam attacks

From: Paul Judge <[email-masked]>
Date: Thu, 31 Jul 2003 17:09:34 -0400

Thanks Peter. This is the appropriate attitude. 

He has put forth some very preliminary data. He is requesting input towards
a better analysis. Those that have constructive feedback please provide it.
There is no point in arguing over the format or making rude criticisms when
he already stated that this is very preliminary.

Ok, what else would be interesting to see?

A broader set of test cases ( more addresses, different domains, ...)
A control set of email addresses with different systems (no spam detection,
detection with a different response such as: DSN, challenge, etc)
What else?

> -----Original Message-----
> From: Peter Kay [mailto:[email-masked]] 
> Sent: Thursday, July 31, 2003 4:59 PM
> To: Alan DeKok; [email-masked]
> Subject: RE: [Asrg] 2.a.1 Analysis of Actual Spam Data - 
> Titan Key reduces spam attacks 
> 
> 
> On another note, if the group sees value, I'd be happy to 
> facilitate some additional analysis.
> 
> We can start from today and measure forward for as many days 
> as you desire.  Let the group collaborate on the experiment 
> definition/control/stats/etc and we'll do our best to set it 
> up on our part.
> 
> Peter
> 
> > -----Original Message-----
> > From: Alan DeKok [mailto:[email-masked]]
> > Sent: Thursday, July 31, 2003 10:38 AM
> > To: [email-masked]
> > Subject: Re: [Asrg] 2.a.1 Analysis of Actual Spam Data - 
> > Titan Key reduces spam attacks 
> > 
> > 
> > "Peter Kay" <[email-masked]> wrote:
> > > But the numbers are there for everyone to see.
> > 
> >   ?
> > 
> > >> that log file is at: http://www.titankey.com/misc/logs/logPK.xls
> > 
> >   If you're going to put an analysis up on the web for other
> > people to see, it would be preferable to put it up as a web 
> > page, or standard format, so that other people could see it.
> > 
> > > The trend is either up, flat, or down. According to the linear
> > > regression trendline, the trend is down. If there is a 
> statistical 
> > > definition of "significant" then of course lets apply it. 
> > But if we're
> > > going to apply it then lets also apply this trend against
> > ANYONE elses
> > > trend that is not using Titan Key.
> > 
> >   Or someone using an email address from another domain.
> > 
> >   A trend for one user is nice, but it doesn't make a
> > convincing argument.
> > 
> >   Alan DeKok.
> > 
> > _______________________________________________
> > Asrg mailing list
> > [email-masked]
> > https://www1.ietf.org/mailman/listinfo/asrg
> > 
> > 
> > 
> 
> 
> 
> _______________________________________________
> Asrg mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/asrg
> 

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: "Alan DeKok" <[email-masked]>
Date: Thu, 31 Jul 2003 22:28:17 -0400

> a better analysis. Those that have constructive feedback please provide it.

  "More data, please!"

> Ok, what else would be interesting to see?

  There was some discussion on this topic a few months ago, but it got
lost in the volume of list traffic.

> detection with a different response such as: DSN, challenge, etc)

  I believe that there are three orthogonal characterizations:

	a) time
	b) recipient domain or name
	c) anti-spam system

  For each combination of (time, recipient, anti-spam), there are two
numbers which should be collected:

	a) total number of messages received
	b) of that, the number of messages determined to be spam

  To avoid political issues, I would suggest that sender-oriented
measurements should be explicitely not asked for, or collected.  I
would also suggest that the anti-spam systems be explicitely not
named, other than as a broad characterization such as
"challenge-response", etc.

  An independent third party should collect these measurements.  I
believe there was a volunteer a few months back, who was waiting for
consensus from the group, before he would start the actual collection
of data.

  The data analysis should be fairly straightforward.  After a cursory
evaluation of the data to avoid political issues, the raw data should
be made available to all.  Multiple independent analyses can then be
done.

  Other measurements which should be done are measurements of SMTP
weirdness which doesn't involve message delivery.  e.g. "connect,
EHLO, disconnect".  These behaviours are closely related to spam, but
in many cases do NOT involve anti-spam systems, as no messages are
delivered.  I believe that the anti-spam systems can then be further
sub-divided into two categories: message-based systems, and
network/SMTP systems.

  Alan DeKok.