Subject: RE: [Asrg] A New Plan for No Spam / Velocity Indicator

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Fri, 25 Apr 2003 20:07:18 -0700

> I am bothered by the facile talk from many quarters of a spam corpus
> of a few thousand messages collected over months or years.  
> Characterizing
> spam in an academically rigorous way is hard and I heard of no serious
> efforts in that direction.  If one doesn't have many current samples
> from many addresses at many different kinds of domains, I do not see
> how one could claim to have more than mildly interesting anectdotes.
> We're talking about a population of 1,000,000,000s of messages sent
> daily to 100,000,000s of addresses.  Any sample of fewer than 0.1% or
> 1,000,000s of messages/day (millions per day!) requires convincing
> arguments that it is representative instead of mere anectdotes.

Vernon, how many people do you think Gallup interviews when they do the
polls for the Presidential elections.

Hint: You do not need a very large sample to have pretty accurate results
even if the population is very large.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: "Jon Kyme" <[email-masked]>
Date: Sat, 26 Apr 2003 08:30:10 +0100

>even if the population is very large.

No, but you do need to sample very carefully and have a very good
understanding of the characteristics of the population you're sampling.
Pollsters doesn't just interview the first 1000 people that the guy with
the clipboard can detain. The sample they use will often have been selected
from a larger sample based on models of the population demographic. And/or
they weight the numbers got from the respondents based on their population
models.

A *very small* sample arbitrarily (or randomly) chosen is not generally
useful.

--

---

From: "Alan DeKok" <[email-masked]>
Date: Sat, 26 Apr 2003 10:35:26 -0400

> understanding of the characteristics of the population you're sampling.

  ... for conclusions with high confidence.

  You can always make blanket assumptions about the characteristics of
the population you're sampling.  While your level of confidence in the
conclusions goes down, you also have the choice of comparing analyses
made under *different* assumptions.

> from a larger sample based on models of the population demographic.

  Exactly.  How do they know how to model the population demographic?
Other surveys, samples, and wild-assed guesses.

  Even if they did interview the first 1000 people they met on the
street, that data would allow them to set bounds on population
behaviour, based on key assumptions and models.

  Alan DeKok.

---

From: "Jon Kyme" <[email-masked]>
Date: Sun, 27 Apr 2003 16:54:15 +0100

> Other surveys, samples, and wild-assed guesses.

Yes, and of course, census data.

> behaviour, based on key assumptions and models.

Maybe so. Ask your nearest statistician how much money would they'd invest
in an enterprise that relied on market research done like that.