Subject: Re: [Asrg] 7. Best Practices - DNSBLs - Article

From: "Chris Lewis" <[email-masked]>
Date: Sun, 07 Sep 2003 21:21:07 -0400

Brad Knowles wrote:

> At 3:57 PM -0400 2003/08/12, Chris Lewis wrote:
> 
>>  We know _exactly_ how each of the DNSBLs we're using work - their
>>  effectiveness ratings, FP ratings, because we see and measure all of it.

Catching up...

>     Not true.  You know how the first one in your list is operating, at 
> least to the level of rejections that result from it, and then 
> complaints that come when when those rejections appear to be inaccurate.

>     You don't know how the next one would have done, if the first one 
> hadn't blocked that message.  You don't know how many false negatives 
> made it through.  And of the positives, you don't really know how many 
> are true and how many are false.

Obviously, you don't know how our filtering or our metrics work.

We know exactly how many emails each BL would stop on its own and 
exactly which BLs blocked any given IP.  I've published sets of data 
here to show that.

>     You would have to look up each address in all black lists (noting 
> both hits and misses,

We do.

  as well as the time of the lookup) before making
> your decision, and you would have to use other means to investigate the 
> negatives that make it through the system and re-look them up again at 
> various recurring periods in time (to see if they were added to the list 
> after the message got through your system).

We do that too.

>     Then you would have to track which black lists result in the most 
> positive hits with the lowest false negative and false positive ratios.

We do exactly that.  Except for false negatives.

When you're running multiple filtering methods, false negatives for a 
individual BLs or filtering types are irrelevant.

>     In other words, you'd have to look up every single address that ever 
> contacts your mail server (including ones that may get rejected for 
> other reasons before they would normally hit the black list checks), in 
> all black lists, re-query on a periodic basis, and track all hits and 
> all misses for all IP addresses, ad infinitum.

>     I'm pretty sure you don't do that.

Actually, we do.

For example, I can tell you that 2.5% of the email that got through 
would have been caught except for latency delays in our BLs... ;-)

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Brad Knowles <[email-masked]>
Date: Mon, 8 Sep 2003 08:49:00 +0200

>  would have been caught except for latency delays in our BLs... ;-)

	Do you know what the latency delays are?  For example, if you 
implemented greylisting and enforced a minimum 30 minute delay before 
you allowed them to get through the greylist, for what percentage of 
these messages would that 30 minute delay have been enough for them 
to show up on the BLs?

	Have you incorporated tools like DCC or Razor into your 
methodology?  Do you know how greylisting with a minimum delay time 
(e.g., 30 minutes) would effect DCC and/or Razor, in terms of their 
efficiency?

-- 
Brad Knowles, <[email-masked]>

"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
     -Benjamin Franklin, Historical Review of Pennsylvania.

GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)

---

From: [email-masked] (Justin Mason)
Date: Mon, 08 Sep 2003 09:27:46 -0700

Chris Lewis writes:
>The simple fact of the matter is that open proxy/socks code will _not_ 
>queue - so they won't try a second time[2].  I would strongly suspect 
>that if you made your greylisting timeout _zero_, and simply 400'd the 
>first appearance of a given sender/IP/recipient tuple and accept the 
>next appearance, no matter how quickly, you'd still be getting 90% of 
>what greylisting with a very long timeout would give you.
>
>Of course, spamming tools will evolve, so then you consider increasing 
>the timeouts.  Too far, tho, and it's worse than where you started.  And 
>I don't think you'd ever get to where you'll be able to take into 
>account DNSBL latency.

My opinion is that, if greylisting becomes common, spammers will
simply start saving enough data to perform retries.

After all, a spam message contains

  a) 1 piece of message body text (as a template with $RANDOMIZE
    references etc.), into which these are inserted:
  b) obfuscated email addresses
  c) "random" text

(a) never changes for a given spam run.  (b) never changes for a given
recipient address.  (c) just needs the srand seed to be saved.

That's not a lot of data required to be saved for retries to be 
supported...

> [2] That's not _entirely_ true, I've seen some spammers that retry 550's 
> after DATA several times very quickly (within minutes).  Not sure 
> whether that's proxy or relay behaviour.

Actually, probably broken spamware that's been interrupted/crashed/moved
to another host, without checkpointing which addrs have already been
mailed.  I regularly get duplicated spams to the same address multiple
times in 1 4-hour interval.

--j.

---

From: "Chris Lewis" <[email-masked]>
Date: Mon, 08 Sep 2003 12:44:33 -0400

> simply start saving enough data to perform retries.

Oh, yes, certainly, they can easily do that.  Even with a full blown MTA 
queueing the whole thing.  However, greylisting puts a severe damper on 
total throughput, which may often be enough to tilt the economies of 
scale against it being profitable for most spammers.

> times in 1 4-hour interval.

Actually, I'm referring to "retries" from the same originating IP a few 
seconds apart.

I get lots of duplicated spam from different IPs.  You don't really 
think they care whether their distributed spamware sends me 1 or 15 
copies, do you?

---

From: Brad Knowles <[email-masked]>
Date: Mon, 8 Sep 2003 18:17:58 +0200

At 10:21 AM -0400 2003/09/08, Chris Lewis wrote:

>  We're considering greylisting as an adjunct to our filters.  However,
>  since we have 8 inbound gateways, it could get rather messy.  A
>  simple-minded implementation with a half hour delay would have a four
>  hour worst-case delay...  Not acceptable.

	Whereas a place like AOL (which had over 45 inbound gateways at 
the time I left) would be much, much worse.  This is why places with 
multiple MXes should implement a shared central database for the 
greylisting, so that all machines can benefit from the traffic seen 
by the others.

>  Due to how our DNSBL system works (we do our own zone downloads),
>  and with a eye to not bashing the DNSBL originating servers too much,
>  the latency (for 3rd party DNSBLs) averages around 12 hours.  So,
>  greylisting-delay-for-DNSBL-catchup isn't going to help (us).

	Indeed.

>  The simple fact of the matter is that open proxy/socks code will
>  _not_ queue - so they won't try a second time[2].  I would strongly
>  suspect that if you made your greylisting timeout _zero_, and simply
>  400'd the first appearance of a given sender/IP/recipient tuple and
>  accept the next appearance, no matter how quickly, you'd still be
>  getting 90% of what greylisting with a very long timeout would give you.

	Fair enough.  But then you get people like Vernon Schryver who 
complain that you're not compliant with RFC 2821 if the sender 
doesn't implement a minimum thirty minute retry period, despite being 
unable to provide a reference for this requirement.

	At the very least, I'd like to see some testing showing how much 
greylisting minimum required latency helps, or fails to help.

---

From: [email-masked] (Justin Mason)
Date: Mon, 08 Sep 2003 09:49:20 -0700

> scale against it being profitable for most spammers.

Given the massive increases in spam volume over the last few years, I
think the use of proxies and trojanned machines seems to be increasingly
insulating them from bandwidth expenses.  (IMO)

> Actually, I'm referring to "retries" from the same originating IP a few 

interesting!

> think they care whether their distributed spamware sends me 1 or 15 

No, they certainly do not.

--j.

---

From: "Chris Lewis" <[email-masked]>
Date: Mon, 08 Sep 2003 13:09:07 -0400

>     Whereas a place like AOL (which had over 45 inbound gateways at the 
> time I left) would be much, much worse.  This is why places with 
> greylisting, so that all machines can benefit from the traffic seen by 
> the others.

"Simple-minded" was intended to mean "without a shared central database".

H'm, t'would be amusing to try this out with a much-hacked DNS server 
doing the "sharing".  Create a query like 
"sender.ip.recipient.greylistzone", and let the DNS server do the counting.

> complain that you're not compliant with RFC 2821 if the sender doesn't 
> implement a minimum thirty minute retry period, despite being unable to 
> provide a reference for this requirement.

I'm not the sender, only the recipient.  So, I couldn't be in violation 
of such a rule.

As if spammers care whether Vernon yells at them or not ;-)

> greylisting minimum required latency helps, or fails to help.

This is something I mean to try on our huge spamtrap.  If only as a 
pure-research project...

---

From: Brad Knowles <[email-masked]>
Date: Mon, 8 Sep 2003 20:43:30 +0200

>  "sender.ip.recipient.greylistzone", and let the DNS server do the

	Interesting indeed.  Specifically with regards to the sample 
greylist implementation that is shipped in the latest snapshots of 
postfix, I was thinking about using a shared central MySQL database 
tied to a perl hash, or maybe a local proxy to a central program 
backed with a Berkeley db database tied to a perl hash.

	On a more general level, I was also thinking about how you could 
implement something like this with a dynamic update to a DNS zone.

>  I'm not the sender, only the recipient.  So, I couldn't be in
>  violation of such a rule.

	Anyone who retransmits within the thirty minute timeframe, for 
any reason, gets lambasted.

>  This is something I mean to try on our huge spamtrap.  If only as a

	I look forward to hearing your results.

---

From: Brad Knowles <[email-masked]>
Date: Mon, 8 Sep 2003 20:28:52 +0200

>  That's not a lot of data required to be saved for retries to be

	Of course, but doing retries virtually guarantees that the IP 
address in question (and the message body) would get added to black 
lists and body content hash tables (like DCC and Razor), which would 
seriously reduce their success rate.

---

From: "Peter J. Holzer" <[email-masked]>
Date: Mon, 8 Sep 2003 21:40:42 +0200

> any reason, gets lambasted.

You mean during the "blacklist" time? The sender just gets another
4xx reply and will retry later. At one of the next queue runs it will
hit the "greylist" window and the mail will get through and be entered
into the whitelist.

I'm more worried about senders with multiple outgoing MXs. I've
already[0] observed one mail which took about 16 hours to be accepted,
because the IP adresses of the sender changed apparently at random, so
greylist entries expired several times until it managed to hit the
greylist window with the same sender IP.

	hp

[0] 2 weeks on a box with only a few mailing lists and no real users.
Maybe a few dozen incoming mails per day.

-- 
   _  | Peter J. Holzer    | Humor ohne Emoticons ist trockener Humor.
|_|_) | Sysadmin WSR       | 
| |   | [email-masked]         |	-- Toni Grass in aip
__/   | http://www.hjp.at/ |

---

From: Brad Knowles <[email-masked]>
Date: Tue, 9 Sep 2003 00:50:34 +0200

At 9:40 PM +0200 2003/09/08, Peter J. Holzer wrote:

>  4xx reply and will retry later.

	No, in this case the sender gets a personal flame from Vernon.

>  greylist window with the same sender IP.

	Indeed, that can be a serious problem.  As can MTAs that use 
VERPs, since the envelope sender address will be different every time.

	Greylisting most definitely is not without its problems.

---

From: "Chris Lewis" <[email-masked]>
Date: Mon, 08 Sep 2003 21:35:55 -0400

>     No, in this case the sender gets a personal flame from Vernon.

Izzat an RFC requirement? ;-)

Saves me having to flame them (remembering one IP that retried a single 
email message over two _million_ times in less than a day before I 
phoned him and told him to knock it off.)

>     Indeed, that can be a serious problem.  As can MTAs that use VERPs, 
> since the envelope sender address will be different every time.

I need to read up on VERP.

>     Greylisting most definitely is not without its problems.

As with many things, often you simply end up with an approximation.

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 08 Sep 2003 23:54:09 -0400

> simply start saving enough data to perform retries.

The longer a spammer keeps using a specific IP address, the bigger is 
the chance of him getting traced by the ISP or being kicked off by the 
ISP. In general, Greylisting would force spammers to use specific IP 
addresses for longer periods of time, killing the advantage that they 
gain by moving from server to server.

Yakov

---

From: "Chris Lewis" <[email-masked]>
Date: Mon, 08 Sep 2003 10:21:14 -0400

> these messages would that 30 minute delay have been enough for them to 
> show up on the BLs?

Due to how our DNSBL system works (we do our own zone downloads), and 
with a eye to not bashing the DNSBL originating servers too much, the 
latency (for 3rd party DNSBLs) averages around 12 hours.  So, 
greylisting-delay-for-DNSBL-catchup isn't going to help (us).

Most of the third party DNSBLs have an average latency on the order of 
an hour or two (or considerably longer) when querying them directly[1]. 
  So, even by switching to real-time queries, it still wouldn't help 
greylisting (unless you were willing to deal with that much longer 
latencies). Only spamcop is any faster.  And SpamCop we won't use, 
because its FP level is too high (for a organization like us).

That being said - I don't think it's necessary to tune your greylisting 
timeout w.r.t. DNSBL latencies.

> (e.g., 30 minutes) would effect DCC and/or Razor, in terms of their 

[1] Do also consider that our 2.58% number is based over a 14 _day_ 
interval.  There's no way of telling the distribution of that DNSBL 
"catchup" over the 14 days.  I don't think the DNSBL latencies on 
individual IPs is anywhere near close to the latency of update. Ie: 
there's no particular reason why a given IP hits the third party DNSBL 
detector at the same time it first hits you.  It takes time (as much as 
hours) for decent/"polite" open relay/proxy testing.

So, short of perhaps the CBL (and SpamCop of course) which doesn't do 
server testing, it's not possible to get latency down _that_ short even 
for direct query.