Subject: Re: [Asrg] Fwd: Returned mail: see transcript for details

From: "Jon Kyme" <[email-masked]>
Date: Tue, 01 Apr 2003 09:26:23 +0100

> Vernon Schryver <[email-masked]> wrote:
> > As others have pointed out, there is a genuine technical ambiguity in
> > "false positive."  Is it RD/TOTAL or RD/SPAM?  (for RD=rejected but
> > desired by recipient, TOTAL=total mail of all sorts whether rejected
> > or not, and SPAM=whatever that means).
> 
> 
>   As an absolute number, false positive should be easy to define.  "I
> went through my 'spam' folder, and I decided that 3 messages should
> not have been put there."
> 
>   As a ratio or percentage, false positives are taken relative to the
> desired signal, not to the total data, or to the noise.  So we have:
> 
>   MAIL   = number of emails you want to be marked as ok (not spam)
>   JUNK   = number of emails you want to be marked as spam
> 
>   MARKED = number of emails actually marked as ok
>   SPAM   = number of emails actually marked as spam
> 
>   FP     = false positives (number of "MAIL" in "SPAM", which should
>            have been in "MARKED")
>   FN     = false negatives (number of "MAIL" in "MARKED", which should
>            have been in "SPAM")
> 
>   The ratio of false positives = FP / MAIL
>   The ratio o false negatives  = FN / MAIL
> 
>   MARKED = MAIL - FP + FN
>   SPAM   = JUNK + FP - FN
> 
>   Alan DeKok.
> _______________________________________________
> Asrg mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/asrg
> 
> 

I wonder if it might be useful to adopt the widely used ideas of 
specificity and sensitivity.

so if 
TP = number of emails you want to be marked as spam that actually are so
marked
and 
FP = number of emails marked as spam that shouldn't have been
and
FN = number of emails not marked as spam that should have been

specificity = TP/(TP + FP)
sensitivity = TP/(TP + FN)

(I think I've got that right - I'm sure there are lot's of references out
there)
These measures are commonly used for comparing diagnostic tests in clinical
applications.

--
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: "Jon Kyme" <[email-masked]>
Date: Tue, 01 Apr 2003 15:17:28 +0100

Correction - apologies.

>These measures are commonly used for comparing diagnostic tests in

Oops, no, I don't think that's right - it should be:
Specificity = TN/(TN+FP)

positive predictive value = TP/TP+FP

It's been a very long time since I used this part of my brain.

Isn't this all used in computer vision and other AI stuff?

Has anyone got a good reference?

---

From: "Alan DeKok" <[email-masked]>
Date: Tue, 01 Apr 2003 11:19:27 -0500

> Isn't this all used in computer vision and other AI stuff?

  Lots.  Medical physics, particle physics, or anywhere else that
needs data analysis.

> Has anyone got a good reference?

  A quick search yields:

http://www.musc.edu/dc/icrebm/diagnostictests.html

  It talks about 'disease' instead of 'spam', but I'm willing to live
with that re-definition of 'spam'...

  Alan DeKok.