Subject: Re: [Asrg] SICS

From: [email-masked]
Date: Sat, 25 Dec 2004 03:08:25 +0000 (GMT)

>> There is clearly NO (local) solution for DDOS attacks, because by
> the time they arrive at your entrance portal the damage has already
> been done.

> Available information before that point is limited. You can refuse to
accept the connection based on IP address, or you can accept the
connection tentatively and do some cheap lookups. Before the message
DATA arrives, you have three pieces of information: a HELO (or EHLO)
identification string, the MAIL and RCPT strings (There's also VRFY or
EXPN but keep it simple).  Once the DATA command is used, the
connection is no longer cheap, because the data can be arbitrarily
long, and doing any sort of analysis on it will cost unpredictable
resources anyway. So discussing body analysis is irrelevant for this
problem.

Maybe, but HEADER analysis might be helpful.

After the receiving mail server has gotten the complete full header, then it 
knows not only who the intended recipient is but also who the stated sender is, 
and that could permit it to establish what that recipient's permissions are for 
that sender... and SOME of those, at least, could perhaps be enforced during the 
time the incoming mail arrival is being done.  

The easiest one to enforce is maximum allowed message size for that sender (or 
default 'unknown' sender) since that one already has to be done anyhow if the 
user's maximum inbox size is exceeded (and that isn't known, either, until too 
much data comes in).  

It's a little more complicated (but not HUGELY difficult) to have the receiving 
server recognize message divisions and enforce (at least on a preliminary, 
tentative basis) recipient permissions for attachments, HTML-burdened 
attachments, and such.

But (although there is a potential cost savings from truncating such mails 
earlier in the process, before they have been fully transferred) there is a 
downside which may override the potential savings... and that is the recipient's 
desire to be able to change their mind upon considering the "spam" filtering 
decision... it's harder to do that, and go ahead and approve the message for 
delivery (and possibly revising the filtering rules for that sender), if in fact 
it was blocked or the server connection was dropped during transfer.  It may be 
that the bandwidth cost (and that's ultimately getting cheaper and cheaper) is 
simply less costly than the added complexity of trying to be more clever, "too" 
early in the process.

Gordon Peterson                  http://personal.terabites.com/
1977-2002  Twenty-fifth anniversary year of Local Area Networking!
Support free and fair US elections!  http://stickers.defend-democracy.org
12/19/98: Partisan Republicans scornfully ignore the voters they "represent".
12/09/00: the date the Republican Party took down democracy in America.

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Devdas Bhagat <[email-masked]>
Date: Sat, 25 Dec 2004 13:20:51 +0530

> Maybe, but HEADER analysis might be helpful.

I don't think I quite understand your point here.

A mail transaction typically goes like this:
<tcp connection> from high port to port 25 of the server
The server checks up the reverse DNS of the client.
S: 220 <server.hostname> [ESMTP]
C: HELO <client.hostname> (or EHLO for ESMTP)
S: 250 Ok
C: MAIL FROM:<[email-masked]>
S: 250 Ok
C: RCPT TO:<[email-masked]>
S: 250 Ok
... (client adds more RCPT TOs here)
C: Data
S: 354 Send CRLF.CRLF to end
Message headers

Message Body
.
S: 250 Ok
quit
S: 221 Ok
<tcp connection teardown>

Now, you know the actual message size only after the data was sent. If
the client sends a ehlo, it may send a size value, but that cannot be
known to be correct. The message headers and body are part of the
message itself, as part as SMTP is concerned.

> knows not only who the intended recipient is but also who the stated sender
> is, and that could permit it to establish what that recipient's permissions
> are for that sender... and SOME of those, at least, could perhaps be
> enforced during the time the incoming mail arrival is being done.  

This is more complex, but doable. A restrictions lookup table keys off
the recipient address, and 

> 
> The easiest one to enforce is maximum allowed message size for that
> sender (or default 'unknown' sender) since that one already has to be
> done anyhow if the user's maximum inbox size is exceeded (and that isn't
> known, either, until too much data comes in).  

Actually, this isn't known until delivery is attempted. No one needs to
have the mail storage system on the same server as the receiving MTA.
The complexity and fraility of trying to push quota overflow conditions
in realtime to external MTAs is bad enough that I would just allow for
bounces to be generated later.

> 
> It's a little more complicated (but not HUGELY difficult) to have the
> receiving server recognize message divisions and enforce (at least on a
> preliminary, tentative basis) recipient permissions for attachments,
> HTML-burdened attachments, and such.

This requires a large amount of code to be pushed into MTAs. Parsing
text takes up CPU. Gobs of CPU. Just to show how much pre data filtering
can help (I wont quote names here, since I don't have permission to do
that yet):
A canadian ISP had 4 boxes handing 200K mails/day each on average, with 
SpamAssassin filtering the content for suspicious mail. Note that this was 
post recipient validation, with a few externsions filtered out
(exe/cpl/pif/.., the common viral vectors). SA was running on 9 boxes,
and those were barely able to keep up with the load (SA was daemonised).
By adding a single DNSBL (the cbl-sbl.spamhaus.org list), they reduced
the inflow to 80K mails/day per host on average, and need 6 SA boxes for
filtering comfortably.

I'll let you figure out the bandwidth and server savings, and the
administrative time saved on that (the admins had been trying for three
months at least to tune the SA boxes so that they would keep up).

> downside which may override the potential savings... and that is the

When do you plan to show actual savings? There are no savings post DATA,
except the cost of putting that message in the temporary mailstore on
the server.

> recipient's desire to be able to change their mind upon considering the
> "spam" filtering decision... it's harder to do that, and go ahead and
> approve the message for delivery (and possibly revising the filtering

Have you considered that message may have more than one recipient, and
that enforcing different message sizes per recipient in the MTA is simply 
not feasible (there is exactly one return value after data)?

> rules for that sender), if in fact it was blocked or the server connection
> was dropped during transfer.  It may be that the bandwidth cost (and 
> that's ultimately getting cheaper and cheaper) is simply less costly than
It is? Not as fast as the spam volume is going up.

> the added complexity of trying to be more clever, "too" early in the
> process.

Devdas Bhagat

---

From: Laird Breyer <[email-masked]>
Date: Sun, 26 Dec 2004 11:42:05 +1000

On Dec 25 2004, [email-masked] wrote:
> 
> Maybe, but HEADER analysis might be helpful. (snip)

I fail to see how any of these points address the specific issue being
discussed. A header is part of the DATA, it can be arbitrarily long
and arbitrarily complex to parse. You simply cannot expect to be able to
allocate a fixed amount of resources of all types if you intend to 
read a header (or any number of body lines for that matter).

Moreover, being part of the DATA, if you intend to accept a single
header line, chances are you'll receive the whole message anyway due
to buffering, and what are you going to do with the full message? If
you intend to analyse it, it will be simpler overall to accept a connection
directly through a full SMTP server.

The only way to make the rejection of obvious invalid requests truly
scalable is if you can guarantee a small fixed cost in CPU cycles,
RAM, I/O. Then you have to run this on a dedicated system which
doesn't do unpredictable things like swapping memory pages, spawning
processes, etc. It has to work like a self contained appliance basically, 
only then will you be sure that the resources used are proportional to
the number of incoming requests, ie it's scalable. That also means no
talking to databases, DNSBLs etc.

A mail system with this kind of front end still has to contend with
too much spam, but I think we can then discuss per user spam volume rather
than incoming SMTP bound volume. Making that point was my only aim in
this thread.

Message analysis has its place somewhere after the message is
accepted, but does nothing at all to reduce the SMTP load.

-- 
Laird Breyer.