Subject: Re: False Positive -was- RE: [Asrg] Fwd: Returned mail: see transcript for details

From: Liudvikas Bukys <[email-masked]>
Date: Tue, 01 Apr 2003 14:18:34 -0500

I slightly prefer the terms accept/reject over negative/positive.

PROPORTION DEFINITION:

I think that a 2x2 matrix is most straightforward presentation:

	{accept,reject} x {ham,spam}

	accept	reject	total
ham	TA	FR	NHAM = TA+FR
spam	FA	TR	NSPAM = FA+TR
total	TA+FA	FR+TR	NTOTAL

and the most helpful intuitive proportions (I think) would be
FRp = FR/NHAM and FAp = FA/NSPAM.

Numbers will ALWAYS be dependent on a particular corpus.

However, I think that the definitions above will be stable
(measuring classifier quality, not corpus composition) over
a wide range of ham-spam ratios.  Using FR/NTOTAL or FA/NTOTAL
will be dominated by ham-spam ratio of the test set, obscuring
the performance of the classifier, making results unnecessarily
corpus-specific.

OTHER MEASURES:

Precision/recall curves are another candidate.

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Clifton Royston <[email-masked]>
Date: Wed, 2 Apr 2003 10:45:01 -1000

> the performance of the classifier, making results unnecessarily

  This is exactly correct, IMHO.

  Your matrix is a flipped version of the matrix I just presented in
proposing the FP/FN calculations to be used for the "dSpam" measure.

  -- Clifton

-- 
     Clifton Royston  --  LavaNet Systems Architect --  [email-masked]

  "If you ride fast enough, the Specialist can't catch you."
  "What's the Specialist?" Samantha says. 
  "The Specialist wears a hat," says the babysitter. "The hat makes noises."
  She doesn't say anything else.  
                      Kelly Link, _The Specialist's Hat_
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Jim Youll <[email-masked]>
Date: Wed, 2 Apr 2003 15:56:09 -0500

I love statistics, but is it possible that not-spam could be possibly 
called "not spam" rather than "ham" in the research-and-report 
context? The word "spam" creates enough difficulty on its own without 
adding another "zany techie word."

---

From: Matt Sergeant <[email-masked]>
Date: Thu, 3 Apr 2003 14:46:28 +0100

> adding another "zany techie word."

"Ham" is in very common use now in the anti-spam community. I don't see 
any valid reason to stop using it.

Matt.

---

From: Jim Youll <[email-masked]>
Date: Thu, 3 Apr 2003 09:41:37 -0500

At 14:46 +0100 4/3/03, Matt Sergeant wrote:
>>I love statistics, but is it possible that not-spam could be 
>>possibly called "not spam" rather than "ham" in the 
>>research-and-report context? The word "spam" creates enough 
>>difficulty on its own without adding another "zany techie word."
>
>"Ham" is in very common use now in the anti-spam community. I don't 
>see any valid reason to stop using it.

It's a cute, meaningless, trite word.

Do you want to be taken seriously, or just use the latest overly 
clever geek phrase
because you can?

I don't see this sort of behavior in any other serious research 
community, and for
that matter other than the April 1 RFCs, IETF work generally has 
always been a fairly
sober affair using precise language rather than clever language.

---

From: Matt Sergeant <[email-masked]>
Date: Thu, 3 Apr 2003 16:06:44 +0100

> It's a cute, meaningless, trite word.

And "spam" is named after a monty python sketch.

"Ham" was used by almost every single presenter at the MIT conference. 
It's a recent term, but it's very much in use. Language changes, 
welcome to the world :-)

Matt.

---

From: Jim Youll <[email-masked]>
Date: Thu, 3 Apr 2003 10:13:55 -0500

At 16:06 +0100 4/3/03, Matt Sergeant wrote:
>>>"Ham" is in very common use now in the anti-spam community. I 
>>>don't see any valid reason to stop using it.
>>
>"Ham" was used by almost every single presenter at the MIT 
>conference. It's a recent term, but it's very much in use. Language 
>changes, welcome to the world :-)

Rrrright.
Fine.
Go for it.

I have to remind people like you, from time to time, that books like 
_Neuromancer_ are works of fiction. If you want to fit into the real 
world (which was 99.99999% not at the MIT conference) try to 
communicate like regular people do.

This thread should be ended now. Do whatever you want, but the 
"cause" still sounds like a plaything when it gets into the business 
of making of cute words that carry no inherent meaning (don't forget 
the endless, years-long debate about "what is spam" that could easily 
be ended if people would just qualify what the hell they are talking 
about rather than trying to tack a name to it)

Even if the word is "okay" to use, it's too broad, isn't it. There 
are myriad little problems, not one big problem. I'm properly 
suspicious of any "research" who presents a metric claiming to 
measure either "spam" or "ham" rather than clearly stating what 
criteria were used to classify the results.

---

From: Matt Sergeant <[email-masked]>
Date: Thu, 3 Apr 2003 17:13:26 +0100

On Thursday, Apr 3, 2003, at 16:13 Europe/London, Jim Youll wrote:

> Even if the word is "okay" to use, it's too broad, isn't it. There are 
> myriad little problems, not one big problem. I'm properly suspicious 
> of any "research" who presents a metric claiming to measure either 
> "spam" or "ham" rather than clearly stating what criteria were used to 
> classify the results.

Let me re-state my position on the use of "ham". It's a cutesy word (in 
the same way that spam is). It shouldn't be used in published materials 
(in the same way that "spam" shouldn't be). This list is not what I 
would consider published materials.

My position is that while the word is ambiguous and trite and cutesy 
it's a really huge waste of time and use of this list getting annoyed 
at people for using it. There are far more important things to worry 
about.

This is my last post about "ham". Mail me privately if you wish to 
discuss it.

Matt.

---

From: matthew richards <[email-masked]>
Date: Thu, 3 Apr 2003 10:15:50 -0800

i agree. "ham" just sounds stupid. the context is vague, the meaning 
is ambiguous. i have nothing against colloquialisms, but seriously... 
ham? you've got to be kidding. we should use an acronym instead, 
something along the lines of SVN (signal verses noise) or AIM 
(acceptable interpersonal mail) etc. im sure someone can come up with 
a good one.

---

From: Kee Hinckley <[email-masked]>
Date: Thu, 3 Apr 2003 17:46:57 -0500

At 10:15 AM -0800 4/3/03, matthew richards wrote:
>is ambiguous. i have nothing against colloquialisms, but 
>seriously... ham? you've got to be kidding. we should use an acronym 
>instead, something along the lines of SVN (signal verses noise) or 
>AIM (acceptable interpersonal mail) etc. im sure someone can come up 
>with a good one.

Human-Acceptible Mail?
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

I'm not sure which upsets me more: that people are so unwilling to accept
responsibility for their own actions, or that they are so eager to regulate
everyone else's.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: matthew richards <[email-masked]>
Date: Fri, 4 Apr 2003 09:21:09 -0800

why not: B.A.C.O.N. (bad aggravating commercial offensive nonsense) 
;) i'm joking of course.

>>i agree. "ham" just sounds stupid. the context is vague, the 
>>meaning is ambiguous. i have nothing against colloquialisms, but 
>>seriously... ham? you've got to be kidding. we should use an 
>>acronym instead, something along the lines of SVN (signal verses 
>>noise) or AIM (acceptable interpersonal mail) etc. im sure someone 
>>can come up with a good one.
>
>Human-Acceptible Mail?
>--
>Kee Hinckley