Subject: RE: [Asrg] Requirements for gathering statistics

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Mon, 24 Mar 2003 09:07:27 -0800

I think that the discussion of accuracy is misplaced. I don't think
we need to measure to n decimal places here. In fact I don't think
we need any decimal places.

We have two pieces of information that I think most significant:

1 Spam is growing at an exponential rate (10% a month by the most
conservative estimates).

2 The spam senders are using countermeasures to bypass filtering.

The types of information I think IS interesting is information that
gives a handle on the character of the problem. In particular

  - Proportion that comes from list kiddies (small fry who bought 
	a list) vs garbage creators (major spamhaus)

  - Rates of growth of spam from different sources.

  - Whether certain countermeasures have been applied in more than
	5% of cases.

  - Proportion of spam sent through open relays - to 10% or so.

I am not interested in the precise figures as they will change. We
are dealing with an enemy that will change its behavior in response
to our own.

Clearly there is an s curve here, there is a limit on the exponential,
however 1) I don't know where it is, 2) I have no way of estimating
where it might be and 3) I suspect it depends on us comming up with
a countermeasure.

If no spam sender ever uses forged headers we could cut out a lot
of spam simply by whitelisting for the major ISPs known to implement
rate limiting. I doubt that such will remain the case for very long.

		Phill

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: "Alan DeKok" <[email-masked]>
Date: Mon, 24 Mar 2003 09:57:45 -0500

> 1 Spam is growing at an exponential rate (10% a month by the most

  I would like additional data justifying that opinion.  While it's
true anecdotally for most people, I think we need a larger sample to
state it decisively.  (This is also known as "Dave Crocker wasn't
entirely wrong...")

  I would like ASRG to be able to say "we looked at the problem, and
some thousands of MTA administrators reported X amount of
spam. Stastical validy is <foo>.  Data analysis follows.  Therefore,
to address these problems, we have potential solutions A, B, and C.
The cost of these solutions is predicted to be D, E, and F.  The
effectiveness of these solutions is predicted to be G, H, and I."

> 2 The spam senders are using countermeasures to bypass filtering.

  Those are difficult to measure.  We *know* some spam is caught, so
that's an easy number to measure

> 	a list) vs garbage creators (major spamhaus)

  I'm not sure how I would tell the two apart, which is why I didn't
ask for those statistics.

>   - Proportion of spam sent through open relays - to 10% or so.

  That involves asking ISP's to determine which IP's are open relays,
which is difficult.

  The questions to blacklist maintainers would be more likely to get
this kind of information.  e.g.:

  - # of IP's determined (somehow) to be open relays
  - # of MTA's using the blacklist
  - # of queries to the blacklist
  - % of queries which return "in the blacklist"

> rate limiting. I doubt that such will remain the case for very long.

  If spammers rate limited their sending to no more than some small
percentage of an ISP's non-spam email traffic, then I would think that
most people would say the spam problem is largely solved.

  Alan DeKok.

---

From: Vernon Schryver <[email-masked]>
Date: Mon, 24 Mar 2003 16:21:09 -0700 (MST)

> From: "Alan DeKok" <[email-masked]>

> state it decisively. ...

10%/month is 2.6X per year.  Spam is increasing but 2.X/year is "most
conservative" only if you also believe "SPEWS blocked all of UUNET."

The number of checksums of spam in the DCC network is increasing faster
than that.  However, I think much of that is due to the appearance of
<!--HTML--> hacks and certain other things and the DCC clients that
still have not installed the countermeasures and so are seeing many
checksums for a few spews.
It's also important to pick durations.  In some weeks spam has increased
by more than 10%, but it has decreased in other weeks (e.g Thanksgiving
and Christmas/New Years)  (again, judging from the number of checksums
in the DCC network)

> ...

That's going to be hard because so many commentators assume the
inevitability of miraculous deployment.  Even those who agree that
replacing everything is unlikely sometimes slip into claiming that
dividing the deployment problem into stages makes it easy. 
(Counterproofs:  IPv6 and DNSSEC)  You can't really talk about the
quantitative effectiveness of solutions that are as likely to be deployed
and work in the real world as spammers getting religion and stopping
on their own.

] From: Dave Crocker <[email-masked]>

] I would appreciate your clarifying your comments.  My confusion is
] between survying attitudes vs. surveying behaviors.
]
] Survey research is excellent for assessing people's attitudes ...

] Survey research is very nearly useless for assessing people's actual
] behaviors. ...

surveying for measurements of relatively abstract numbers like
"total spam" is even more problematic.

Finding representative samples is hard, because spam is so idiosyncratic.
The numbers at http://www.dcc-servers.net/dcc/graphs/comp-rates
show a 2X variation in spam load for some types of oganizations, or at
least some particular outfits.
Note that the names of those outfits are intentionally unstate.  Most
organizations seem to be unenthusiastic about publishing such numbers.

I like seeing numbers that are based on millions of mail messages,
but even they must be viewed as special cases of the global situation.
The more common numbers of the "my personal mailbox has seen X" are
rarely interesting.

Vernon Schryver    [email-masked]
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg