Subject: Re: [Asrg] where the message originated

From: Gordon Peterson <[email-masked]>
Date: Tue, 13 Jan 2009 19:13:39 -0600

> But none of this is necessarily true.

IF the mail is bounced because it contains a virus or worm, and if that 
virus or worm is KNOWN to falsify return addresses, then I claim that 
there is little or no value in bouncing it... because it is not clear 
that ANYONE working backwards is going to be able to figure out reliably 
who (if anybody!) should be notified of the problem.

It doesn't really much matter which antivirus software identified the 
message as malicious (although it would be nice to have an audit trail 
somewhere such that a misbehaving piece of related software can be found 
and fixed).

[snip]

 > - A human being won't necessarily be harmed by this.  For starters,
there's no way to know that the message will be delivered.  If the
putative sender is elsewhere, then the mail system there might
detect and reject the message.  The putative sender's mail client
may be using an anti-virus product that detects it.  Or the putative
sender may never "open" the message (which seems to effectively
defuse most, but not all, of these problems).

That's a bogus argument.  You're basically saying that "someone else 
will probably pick up on this" and using that as a (lame) excuse to lob 
a grenade their way.

-- 

Gordon Peterson II
http://personal.terabites.com
1977-2007:  Thirty year anniversary of local area networking
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: Rich Kulawiec <[email-masked]>
Date: Tue, 13 Jan 2009 20:46:19 -0500

> will probably pick up on this" and using that as a (lame) excuse to lob  

Perhaps you should review the SMTP protocol: sending a 5XX response
*refuses* delivery of a message.  It does not transmit (or retransmit)
a message.  It is difficult to see how one can be accused of "lob[bing]
a grenade" when one has never taken possession of it.

---Rsk

---

From: David Wilson <[email-masked]>
Date: Wed, 14 Jan 2009 09:04:58 +0000

> a grenade" when one has never taken possession of it.

Because of the normal action of an MTA when it receives such an 5XX
response, i.e. it sends a non-delivery message, normally containing the
message, to the return path address. If that return path address is
forged, then the infection is bounced elsewhere. That is the grenade.

---

From: Rich Kulawiec <[email-masked]>
Date: Wed, 14 Jan 2009 07:42:40 -0500

> forged, then the infection is bounced elsewhere. That is the grenade.

First, even if this is the case -- and it's rather unlikely that it is,
as I'll get to in a minute -- it is obviously disengenuous to
accuse the refusing system of being responsible for this situation.

And second, have you (rhetorical you) been paying attention to the
origination points of the vast majority of attempted malware deliveries
over the past decade?  They're coming from places like this (seen within
the last hour here):

	11.red-81-43-100.staticip.rima-tde.net [81.43.100.11]
	12-216-204-70.client.mchsi.com [12.216.204.70]
	122-82-124-91.pool.ukrtel.net [91.124.82.122]
	1809ds4-fb.0.fullrate.dk [90.184.161.72]
	78-0-202-32.adsl.net.t-com.hr [78.0.202.32]
	89-180-255-125.net.novis.pt [89.180.255.125]
	cable-87-116-168-96.dynamic.sbb.rs [87.116.168.96]
	cpe-69-133-71-112.columbus.res.rr.com [69.133.71.112]
	ip-90-187-210-98.web.vodafone.de [90.187.210.98]

Do you think any of those are actual mail servers?  Do you think that
any of them actually are running an SMTP engine which does something
with rejected traffic?  Do you think that any of them are running an SMTP
engine which even *notices* rejected traffic?

Malware authors have long since [mostly] outgrown the need to use existing
mail systems for distribution.  Why should they limit themselves to a
small number of systems (~100K if the Leisi Conjecture is correct) when
they can use hundreds of millions?  And why should they bother coding
elaborate, fully-functional, standards-compliant SMTP engines when
they don't care and have no reason to care about most of that functionality?

Third, you cannot know the state of a system which presents messages
carrying malware.  It might be okay, except that its particular anti-virus
setup did not detect this particular virus at this particular time.
Or it might be fully-compromised, 0wned by the enemy, and repurposed
as a full-time virus distribution engine.  Or anything in between, or
something quite different.  Because you CANNOT know its state, you CANNOT
predict what it will do in response to any SMTP response code you send it.
It *might* generate an NDR.  It *might* ignore it.  It *might* try sending
the same message again.  It *might* crash instantly.  It *might* do all
kinds of other things. [1]

Because you can't know this, it's pointless to speculate, and even more
pointless to arbitrarily choose one possibility among many and then try
to address that one as if it were a certainty.  It's clear that the
best you can do is to refuse the traffic, at which point you've done
your part not just for your own operation, but for the rest of the 'net
as well.  (And optionally, depending on circumstances, refuse *all*
traffic from it, as I do from the hosts above, since it's clear that
100% of everything they will ever present will be spam and malware.)

Fourth, and this veers a bit off-topic, but do realize that not everyone
uses the markedly inferior operating systems which are routinely compromised
via malware authored by mere children.  I consider any operating system
which requires anti-malware software to survive in the wild to be seriously
broken and therefore unworthy of any further consideration.  ("The brakes on
this car don't work, but you can buy this add-on which will patch them...")
Yet, out of a sense of responsibility to the greater Internet community,
I've expended considerable time and money designing and deploying
anti-malware measures, NOT for my benefit, because I don't care about
malware and don't have to care, but for theirs.

However, the reach of those measures, like everything else I've done,
extends only as far as the perimeter of my own operation.  That's also as
far as my responsibility extends.  I can't affect what happens "out there",
other than perhaps indirectly via my limited power of persuasion,
and I certainly can't be held accountable for it.

On the other hand, who can we hold accountable for

	cpe-69-133-71-112.columbus.res.rr.com [69.133.71.112]

and the other systems like it?  I'll argue:

	a) the putative (former) owner whose desk or lap it sits on
	b) the new (real) owner(s) who have repurposed it
	c) RoadRunner's network engineers
	d) the vendor of the operating system it's running (do I even need
		to mention which one passive OS fingerprinting indicates?)

So if we must get into a policy debate about responsibility, I'll be
happy to engage in discussion about which of a-d should be at the front
of the line, and which should bring up the rear.

---Rsk

[1] In the typical case of zombies like the ones I listed above, the
treatment of *any* SMTP response code is "ignore it".  The SMTP engines
installed on such systems are designed to maximize attempted deliveries
per unit time, and one excellent way to do that is to ignore everything
sent back by the recipient MX.
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: David Wilson <[email-masked]>
Date: Wed, 14 Jan 2009 13:00:32 +0000

On Wed, 2009-01-14 at 07:42 -0500, Rich Kulawiec wrote:
> origination points of the vast majority of attempted malware
> deliveries over the past decade? 

The error here is to assume that you get the message from the
originating system. There are scenarios in which you get the message
from an innocent (perhaps in more than one sense of the word)
*intermediate* MTA. It is the reaction of such an MTA to a rejection
which is the problem.

An intermediate MTA is not be a open relay, because relaying is often
defined in terms of the system from which the message arrives. For
instance, a company has a "boundary" MTA. A PC within the company's
network is compromised (the user accessed an unsuitable web site in
their lunch break). The trojan sends messages via the boundary MTA.
Unless that MTA checks the return-path address, and the boundary MTA
picks up on the virus, then the rejection by the next hop MTA is a
problem.

After all, if the "MTA" from which you received the infected message is
not innocent, perhaps not even a proper MTA, then rejecting the message
is also pointless. The rejection will be ignored, and so the overall
effect will be the same as if it were accepted and discarded.

---

From: Rich Kulawiec <[email-masked]>
Date: Wed, 14 Jan 2009 08:45:30 -0500

> *intermediate* MTA. It is the reaction of such an MTA to a rejection

That's not an assumption, it's a fact. If 1.2.3.4 connects to my MX's
and attempts delivery, then 1.2.3.4 IS the originating system, as far
as my MX is concerned.

Now the message may -- or may not -- have *really* originated on 1.2.3.4,
but that's of course quite impossible to know.  (I trust everyone is aware
that malware routinely adds fake headers, so anything it claims about
itself, including its putative origin, shouldn't be believed.  The only
things you can know about it are things that you independently determine.)

> picks up on the virus, then the rejection by the next hop MTA is a

So we are to presume that malware which has taken over user U's PC,
and thus is likely already in full control of everything it does,
is going to send a copy of itself through the outbound MTA on that
network, with the goal that it will be rejected by someone's MX,
so that the MTA will generate an NDR and send it back to user U...
who is already infected?  What, exactly, is the point?

Or that this same malware, having rummaged through user U's inbox and
outbox and other files resident on the disk drives on U's PC, has
now decided to attempt delivery to co-workers V, W, and X, and rather
than just attaching itself to the next outbound messages from X to
any of them, or crafting a plausible forgery and sending it now,
will instead try to forge V/W/X's addresses as the putative sender,
submit it to the outbound MTA, and hope that something rejects it
so that the local MTA will send an NDR with it to V/W/X?

We've wandered into fantasyland here.  Why should such elaborate tricks
be used when there are a large number of much easier, eminently successful
tactics to try?  After all, if the MTA doesn't know how to detect this
particular bit of malware when it's headed to an external destination,
then it seems unlikely it'll detect it when it's headed to an internal one.
The simplest course is for it to just send itself directly -- or, even
better, to infest the system running the MTA, which makes further
propagation considerably simpler.

> effect will be the same as if it were accepted and discarded.

*If* it's not a proper MTA, yes.  But you have no way to know that,
nor can you be certain that you're not making a classification error,
so best practice remains to use the SMTP protocol to indicate
rejection.  If it does turn out to be a proper MTA and/or if it does
turn out you've made an error, you've given everyone a decent chance
of noting it, tracking it, and fixing it.

---Rsk

---

From: Paul Russell <[email-masked]>
Date: Wed, 14 Jan 2009 09:04:12 -0500

> forged, then the infection is bounced elsewhere. That is the grenade.

The grenade was not lobbed by the MTA which issued the 5xx response; it was
lobbed by the MTA which previously accepted the message from some source and
attempted to deliver it to the MTA which issued the 5xx response during the
SMTP session.

-- 
Paul Russell, Senior Systems Administrator
OIT Messaging Services Team
University of Notre Dame
[email-masked]
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: Paul Russell <[email-masked]>
Date: Wed, 14 Jan 2009 09:21:47 -0500

> effect will be the same as if it were accepted and discarded.

If I understand your position correctly, you want the receiving MTA to issue a
5xx when the sender is a real mail server, but you want it to accept and discard
the message when the sender is a bot.  As has already been pointed out, for
systems outside your control, you can only speculate as to their true nature and
their likely reaction of a 5xx response.  Why waste time trying to discern the
true nature of the sender, and run the risk that you will discard messages which
should have been rejected, because your analysis of the sender is imperfect?
Just issue the 5xx and be done with it.

---

From: David Wilson <[email-masked]>
Date: Wed, 14 Jan 2009 15:45:32 +0000

> Just issue the 5xx and be done with it.

No, quite the reverse. If you have received an infected message from a
real MTA, then issuing a 5xx response might "do bad things". I.e. that
real MTA might send a DSN containing the infection to a forged returned
path.

What I said was that there was no point in sending a 5xx response to a
bot, since they will ignore it. (I suppose they might remove the
recipient address from their list, but I don't know if this actually
happens.)

So, I believe that sending 5xx to an innocent sender can be dangerous,
and sending 5xx to a bot is pointless.

The only problem with not sending 5xx (or not sending a DSN) could be
with false positives. However, I believe this is a very small problem,
at least for the majority of users, and the danger of using 5xx is
significantly greater.

---

From: "Chris Lewis" <[email-masked]>
Date: Wed, 14 Jan 2009 11:07:41 -0500

> at least for the majority of users, and the danger of using 5xx is

Our experience, issuing 55x at levels of 250,000-1,300,000 per day since
1997 indicates exactly the reverse.

Certainly, there's no way to prove that "it" (an infection) hasn't
happened, but considering we've had precisely _one_ report of a 55x (for
any reason whatsoever) landing in the forgee's mailbox in all that time,
it can't be significant.  In contrast to the dozen or so FPs per day
that we do handle.  Silently dropping emails would mean that those dozen
or so FPs would go unnoticed by anyone.
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: SM <[email-masked]>
Date: Wed, 14 Jan 2009 08:39:41 -0800

>engine which even *notices* rejected traffic?

A few of them may be actual mail servers.  Not all of them may be 
listening for SMTP connections.  A negligible number of them may do 
something about rejected traffic.

Some people find it easier to block SMTP traffic from such 
origination points than to put in the resources to handle them.  If 
we are looking for an identifier which says that "this IP address may 
originate SMTP traffic", then it's better to invent one than to rely 
on hostnames.

Regards,
-sm

---

From: David Wilson <[email-masked]>
Date: Wed, 14 Jan 2009 17:04:29 +0000

> originate SMTP traffic", then it's better to invent one than to rely 

Some DNSBLs have categories which identify IP addresses which are not
expected to originate SMTP traffic.

---

From: David Wilson <[email-masked]>
Date: Wed, 14 Jan 2009 18:00:07 +0000

On Wed, 2009-01-14 at 11:07 -0500, Chris Lewis wrote:
> .

Interesting. At least there are some numbers here.

Just to check, are these 250K-1300K 5xx response *just* for messages
which have fallen foul of Anti-Virus checks (i.e. the number does not
include 5xx responses for other causes like invalid recipients).

As Alessandro Vesely has commented, this does seem a high AV FP rate.

How do you find out about the FP (since the message is, by definition,
rejected)? Obviously, if the sender simply resends, it would be rejected
again. So, there must be some other mechanism you have for finding out
that this particular message was not actually infected.

What kinds of message typically generate FPs? What AV signatures are
being hit by these FPs? How do you verify that the message is not
actually a problem?

Maybe we are lucky, but we do not see many messages which appear to be
infected - perhaps only 0.6% of the overall traffic. Nearly 90% of
received messages appear to be spam. Do you get a significantly higher
proportion of (purported) infected messages?

Although the number of reported infections from rejected messages is
just one, maybe this is not surprising. Firstly, there are clearly "out
there" a very large number of machines which are infected and the owner
either does not know or does not care. So, if any of these have been
infected as a result of one of your rejections, you would not know. Even
if the owner finds out their machine is infected, I would think it is
rare for them to try to trace the origin of the infection. The work
involved in getting their machine clean is probably enough for them. It
might involve re-installing the OS, which would probably wipe the
evidence.

---

From: "Chris Lewis" <[email-masked]>
Date: Wed, 14 Jan 2009 13:58:36 -0500

> include 5xx responses for other causes like invalid recipients).

It's all filter hits, including spam, but not invalid recipients (we
don't, but should, count these at present - at least in production.  On
the traps invalid recipient is 20-30M/day by definition ;-).

That said:

1) That 1.3M/day was virus rejects - the Mydoom explosion in Feb 2006
(IIR the year correctly).  The total rejects for those days was actually
1.6M/day (I rechecked the numbers - was remembering Mydoom's numbers
alone for the total).

2) The FN rate of AV tools these days is atrociously bad.  Studies have
shown that "new" viruses (in this context meaning different MD5
checksums, meaning the same old virus just packed differently) is missed
by 90% of all AV packages, and that only rises to 50% after a month.

In fact, some AV vendors are not bothering to update MD5 databases for
certain viruses, because they're different for every copy.  That's not
supposition, at least one vendor explicitly said so.  In person ;-)

I can't tell you how many times I've submitted a cutwail binary to
virustotal and found that not one of virustotal's battery of detectors
(commercial AV products) fired on it.  Despite cutwail being (a) the 2nd
or 3rd most prolific malcious email sender (peaked at over 20% of all
malicious email), and (b) out there for better than a year.

As a consequence of (2), any non-trivial environment who relies purely
on AV products (even multi-layer such as us, with email and web and
generalized) for virus protection is going to get hosed.  They have to
add in different approaches, such as DNSBL, SURBL, custom patterning and
executable detection.  With _both_ a higher FP rate, and the inability
to tell whether it's a virus or not.  You can't choose to "bounce (or
reject) spam and not viruses" because the filter is unable to make the
distinction.  We used to publish "virus rejects" - we don't anymore,
because we can't distinguish them.

We have an internal formal AV engine in email.  But we don't rely on it
for the heavy lifting of the majority of malicious attacks.  The front
ends use anti-spam techniques, and catch better than 99% of it before
the AV sees (and probably misses) it.

> As Alessandro Vesely has commented, this does seem a high AV FP rate.

The dozen or two FPs is are all a result of rejects, regardless of
actual rule (virus, spam or otherwise)

> that this particular message was not actually infected.\

The rejection message (see previously in this thread for a real example)
states very clearly what the sender should do.  Email the FP handler.
Which is my responsibility.  So I see them all.

> being hit by these FPs? How do you verify that the message is not

Executables that aren't viruses.  Ordinary email coming from infected
machines, or in amongst large numbers of badly infested machines.  Very
badly configured machines.  Etc.

We treat the FP process as simply part of filtering.  If you get a
reject, report it, and we'll fix.  No email lost.  Merely delayed.

> proportion of (purported) infected messages?

I think you have an overly narrow definition of viruses - specifically,
emails that carry malicious executables _directly_.  The trap
demonstrates that, at times, you'll see up to 50% of its entire flow is
viral in the sense of, for example, links to malicious files.

Most AV products probably didn't fire on the MSNBC/CNN viral attack.
Yet, at one point a few months ago, that alone was 25% of all trap
traffic.  That was concurrent with several other bot families expending
a considerable fraction of their output trying to propagate instead of
sending spam.

> infected as a result of one of your rejections, you would not know.

We'd get reports of us being responsible for sending a virus if _only_
from us being mentioned in the rejection notice buried in the email.
Given the sheer volume of people who can't read headers well enough to
identify obviously forged received lines and report the spam to us
because it has a 47/8 address in it, we'd get at least _some_ reports of
us (apparently) bouncing viruses at one point in the past 11 years.

We NEVER have.  The only rejection notice of this type we've ever had
drawn to our attention is when an infected ISP customer sent us a spam
forged in another customer of that same ISP, and our rejection caused
the ISP to send the reject back to the forgee.

Only once in 11 years.  At our volumes, even if off by several orders of
magnitude, the risk of reject-induced-bounce-of-virus is insignificant.
 Even more insignificant when you factor in the cost of missing business
email because you blackholed it because of a filter FP.
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: SM <[email-masked]>
Date: Wed, 14 Jan 2009 11:17:30 -0800

>expected to originate SMTP traffic.

That cannot be standardized.

The usual suspects are already aware of the arguments that have been 
brought up in this thread as they have heard them before.  Even 
though my question about where the message originated wasn't about 
the techniques used to validate the origin, I'll summarize them:

   SPF and SenderID - IP address evaluation (for the long answer, see relevant
   specifications)

   DomainKeys and DKIM - Domain-based evaluation

   DNSBL - IP address evaluation (usually proprietary)

   Hostnames - Reverse DNS evaluation (heuristics)

   Whitelisting - Determined by operational considerations

These techniques are about how to restrict who can talk to us.  At 
another level, we can frame the question as what content is safe and 
relevant to us.  None of the techniques listed above address that.

Regards,
-sm

---

From: Rich Kulawiec <[email-masked]>
Date: Thu, 15 Jan 2009 10:34:47 -0500

> and sending 5xx to a bot is pointless.

But you have no way to tell which is which.  For example: putatively-"real"
mail servers get botted as well.  How should we then treat them?  And
from time to time, a system will turn up that looks much like a bot --
generic rDNS, fingerprints as Windows, etc. -- but turns out to be a
real mail server.

Note as well that 5XX responses do not go to senders: they go to
connecting SMTP clients.  There's no way to know whether or not those
clients will in turn deliver anything to the putative sender.  (Nor
is there any way to know, in the cases where they *do* deliver
something, and in the subset of those cases where this delivery includes
putatively-malignant content, whether or not it's something that's
already long since been delivered to the user by something else.)

> at least for the majority of users, and the danger of using 5xx is

Just the opposite is true here.  Malware is a trifling concern (thanks
to the usage of superior operating systems and mail clients, along with
abundant paranoia), but false positives are an intermittent, and annoying,
problem.  So while I don't doubt that your perspective applies to your
operation, it doesn't apply here: what to some people is a crafty,
dangerous virus that represents an immediate threat is to us merely
a curious collection of bits.

The broader point is that I see no reason why we should change the SMTP
protocol in order to accomodate the thriving malware ecosystem made
possible in large measure by the abysmally low quality of Microsoft
products.  That's akin to putting duct tape over a brightly-flashing
red light because we don't want to know there's a problem.  I think a
much better approach is to attack the problem at its source, or,
failing that, to at least contain the problem at our network perimeters
using the tools we already have.  (And as to my [rhetorical my, mostly]
supposed responsibility to others, I've spent decades discharging that
obligation not just by making it my highest priority to prevent outbound
abuse from anything I touch, whether /32 or /8, but by freely giving
my best professional advice in re the selection of operating systems
and applications.  I can hardly be held at fault by those who have
failed to heed that advice and are now dealing with the very predictable
consequences.

---Rsk

---

From: David Wilson <[email-masked]>
Date: Thu, 15 Jan 2009 17:12:04 +0000

> viral in the sense of, for example, links to malicious files.

What I am talking about is messages for which the content is reported by
AV software as being infected. The probability that such a message is
not harmful is likely to be very, very small. A system should behave
responsibly with such a message, and not risk sending it in a bounce
message anywhere. Rejecting it might result in the generation of such a
bounce, if you have received the message from an intermediate MTA.

---

From: Douglas Otis <[email-masked]>
Date: Thu, 15 Jan 2009 12:19:31 -0800

On Jan 15, 2009, at 9:12 AM, David Wilson wrote:

>> I think you have an overly narrow definition of viruses -  
>> specifically, emails that carry malicious executables _directly_.   
>> The trap demonstrates that, at times, you'll see up to 50% of its  
>> entire flow is viral in the sense of, for example, links to  
>> malicious files.
>
> What I am talking about is messages for which the content is  
> reported by AV software as being infected. The probability that such  
> a message is not harmful is likely to be very, very small. A system  
> should behave responsibly with such a message, and not risk sending  
> it in a bounce message anywhere. Rejecting it might result in the  
> generation of such a bounce, if you have received the message from  
> an intermediate MTA.

Chris Lewis made excellent points about low detection rates with  
today's highly polymorphic threats.  Some providers in emerging  
markets are transparently intercepting traffic in a store and forward  
mode, and then bounce rejections.  Those bouncing the message should  
adhere to RFC 3464,  use the top level content type of multipart/ 
report,  and then minimize the amount of original content.  This makes  
NDNs less inviting as a method to distribute spam, creates less danger  
with respect to malware, and lowers the chances of being identified as  
a source of spam.  False positives can still be determined by the  
sender, but better MUA automation for locating original messages would  
be helpful.

-Doug

---

From: Rich Kulawiec <[email-masked]>
Date: Thu, 15 Jan 2009 18:54:14 -0500

> bounce, if you have received the message from an intermediate MTA.

But this is exactly the opposite of the situation here.  You can
send Windows malware here all day long and nothing will happen except
maybe bemusement: there are no Windows systems here.

"harm" is not an absolute.  Not everyone's environment is the same.

Moreover, trying to guess what someone else's mail system will do is
pointless, and trying to correct it if you think it's wrong is futile.
All you can do is *your part*, which is to reject with an appropriate
error message and move on.

(And as a practical matter, if it's not a real mail server talking to
you, then it's not going to generate a bounce.  If it *is* a real mail
server talking to you, and it's trying to deliver malware, then it's
probably already so badly hosed that nothing you could possibly do or
not do will make it any worse.)

---Rsk

---

From: David Wilson <[email-masked]>
Date: Fri, 16 Jan 2009 10:25:43 +0000

On Thu, 2009-01-15 at 18:54 -0500, Rich Kulawiec wrote:
> What a conforming MTA will do with a 5xx reject from your MTA is well
> defined. From RFC 5321 Section 6.1:

   If there is a delivery failure after acceptance of a message, the
   receiver-SMTP MUST formulate and mail a notification message.

The MTA has accepted the message, and the next-hop MTA has rejected it,
so the message is suffering delivery failure. Therefore the MTA is
obliged ("MUST") to send a notification message.

Therefore, if you always 5xx messages which are known to contain
malicious content, some fraction of those rejections will result in
notification messages which are likely to contain that malicious
content. If the vast majority of malicious content comes from bots or
similar, then the fraction may be small. However, the MTA which
generates the "bounce" is not at fault, it is doing what it is obliged
to do.

> not do will make it any worse.)

An MTA has no obligation to inspect the content of messages which it is
passing through. You might suggest that it does have that obligation,
but there is none. Indeed, inspection of the content of messages by
every MTA might be regarded as a violation of privacy. An MTA is not
"badly hosed" that is simply doing its job of transferring messages
based on the data in the message "envelope" (any more than a postal
worker in a sorting office is at fault for sorting messages which happen
to contain illegal images).

---

From: John Levine <[email-masked]>
Date: 16 Jan 2009 10:50:54 -0000

>content.

 ... and as Chris has told you several times, that fraction has been
observed to be less than 1/1,000,000, orders of magnitude less than
the rejections that alert real users to false positives.  So stop.

Perhaps it's time to amend the ASRG charter to exclude easily
visualized but actually hypothetical threats.  If you disagree that
they're hypothetical, first you have to go get real data to support
your claim.

Regards,
John Levine, ASRG chair

---

From: David Wilson <[email-masked]>
Date: Fri, 16 Jan 2009 11:14:44 +0000

> they're hypothetical, first you have to go get real data to support

So, just to clarify, you are saying that the view of ASRG is that if an
MTA receives a message which when checked using anti-virus software such
as Sophos or ClamAV (note this, my discussion was not about other
anti-spam techniques) give a positive, the correct course of action is
to reject the message using a 5xx response?

---

From: John Levine <[email-masked]>
Date: 16 Jan 2009 11:46:06 -0000

>to reject the message using a 5xx response?

That seems to be the consensus here, yes.

My MTA does rejects on anything that has an attached EXE file, and I don't
ever recall getting any complaints about backscatter.

R's,
John

---

From: Franck Martin <[email-masked]>
Date: Sat, 17 Jan 2009 00:40:48 +1200 (FJT)

It is standard to do that at the end of DATA. And it is best to do it there than later. Whoever is connecting is told "no", rather than sending an NDR to the "mail from" which is likely to be forged.

Sync NDR are better than async NDR.

----- Original Message -----
From: "John Levine" <[email-masked]>
To: [email-masked]
Sent: Friday, 16 January, 2009 11:46:06 PM (GMT+1200) Auto-Detected
Subject: Re: [Asrg] enough about backscatter

R's,
John

---

From: Rich Kulawiec <[email-masked]>
Date: Fri, 16 Jan 2009 09:04:29 -0500

On Sat, Jan 17, 2009 at 12:40:48AM +1200, Franck Martin wrote:
> It is standard to do that at the end of DATA. And it is best to do it there than later. 

Well, you can also do it much earlier.  I reject the overwhelming majority
of mail (>95% of all rejections) before DATA, based on connecting IP,
connecting IP's rDNS, putative sender, recipient, and so on.  "Reject
early, reject often" is one way I've jokingly put it -- after all,
once something has demonstrated that it's actively malicious, there's no
point in even bothering to stick around for the data: 5XX it, hang up,
and move on.

---Rsk

---

From: "Chris Lewis" <[email-masked]>
Date: Fri, 16 Jan 2009 11:10:40 -0500

> point in even bothering to stick around for the data: 5XX it, hang up,

Well, you _could_ reject earlier, but not via a content-based filter,
such as ClamAV.

Secondly, there is occasionally point of sticking around for data, even
if you've already decided you're not going to pass it through to the
recipient.  Your filter could be wrong, and allowing thru to end of DATA
permits you to quarantine, allow the user to see if if they wish via
per-user quarantine, and forward later if required.  Secondly, you can
derive useful information from the body (eg: for other filtering, virus
forensics, samples for virustotal etc)

The first "point" was mandated upon us - the "no lost email" rule.  We
could reject earlier, or even by firewall rule for some situations.
There are tricks with MXes that we've used to completely eliminate some
BOTs managing to connect to us at all.

But at present, we swallow everything thru to end-of-data.
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: "J.D. Falk" <[email-masked]>
Date: Fri, 16 Jan 2009 16:47:41 -0700

> they're hypothetical, first you have to go get real data to support

+1

However, I wouldn't mind discussion about /how/ to get real data....

---

From: Douglas Otis <[email-masked]>
Date: Sun, 18 Jan 2009 17:05:57 -0800

> However, I wouldn't mind discussion about /how/ to get real data....

Some decisions depend upon doubts expressed of potential threats.   
Proponents are likely to insist a threat is only theoretical and do  
little more than make poorly considered comparisons.

Proving a point will likely require staging a vectored attack  
exploiting recipient resources, which seems dangerous from a legal  
standpoint.  Anything less is likely to be dismissed.  MTA  
administrators appear to pay little attention to a problem not  
directly effecting them.  In the meantime, the problem results in a  
greater number of confidence artist victims, and millions of dollars  
in damages occur due to DDoS attacks against those reporting on spam  
or malware threats, for example.  Unfortunately, these attacks rarely  
target MTA administrators who seem willing to look the other way.   
After all, the championed path registration schemes can be structured  
to deflect accountability away from the MTA.  The excuses for doing  
this can be found in appendix D of the Authentication-Results header  
draft.

See:
http://www.ietf.org/internet-drafts/draft-kucherawy-sender-auth-header-19.txt

With the introduction of the Authentication-Results header, the  
underlying goal becomes clearer as a scheme to deflect assessments  
away from the MTA onto that of an authorizing domain.  Basing  
treatment upon an authorizing domain allows an MTA that may suffering  
from compromised access to remain otherwise unimpaired.  The  
Authentication-Results draft never stipulates that the authentication  
results  header should not be added without making a reputation check  
of assumed protections.  Only that the results of this header should  
not be revealed without a reputation check, which included an example  
of a phishing domain.  Of course, no phishing domain should be  
accepted, let alone given an Authentication-Results header right?   
Nevertheless, this check is to be made prior to revealing the content  
of this header, a function likely provided by the MUA.

Those revealing these results are required to assume the authorizing  
domain's reputation had been checked in a manner that directly  
pertains to identity being asserted without the check ever being  
directly indicated within the header.  The draft also states that,  
given the passage of time, rejections (rather than annotations being  
withheld), might occur when MUAs are allowed to ensure these checks  
separately.  This draft seems happy to have checks made against the  
local-part and domain, while also stating that there is a DDoS concern  
regarding the use of the IP address of the SMTP client.  Checks  
against the local-part and the domain represent several orders of  
increased overhead as compared to what is required of the IP address  
of an SMTP client.

Take for example the SPF local-part macros.  These macros are almost  
_never_ legitimately used, and _can_ be exploited by spam campaigns to  
leverage cached SPF records when constructing vectored DDoS attacks  
that do not expend additional resources of the attacker.  It is fairly  
normal to see millions of bot-net sources (IP addresses) becoming  
active for short intervals out of a set of several hundred million  
that may have been dormant for months to years.

Although concern has been expressed about how these macros can be  
exploited, and how impractical it is to use local-part macros to  
produce positive assertions, nothing seems to deter this feature from  
being institutionalized as a means to obtain message annotations as is  
now required by the Authentication-Results header draft!   When one  
wants their email to be noticed, this draft stipulates the use of  
local-part macros as an unintended requirement.

Of course, including the IP address within the Authentication-Results  
header permits a means to mitigate difficult problems that might be  
related to compromised servers or BGP route injections that completely  
defeat path registrations, but this protective information is  
purposely omitted.  The period that separates a message being received  
to when it is handled by the MUA provides valuable time to offer  
protection in the way of annotation.  Instead the draft conflates  
annotation with that of message rejection.  The MailMan program  
running many of the IETF mailing lists handles and annotates messages  
behind the MTA, where the use of RFC 3464 becomes important.  If if  
something more elaborate were to be implemented by the MUA, there are  
solutions in current use to mitigate these threats.  A reputation  
service can easily extend the time of a reporting by indicating the  
number of hours since a threat has been  resolved.  An annotation can  
be applied perhaps just in time, or not depending upon the threat  
information that relates to the actual source of the message, the SMTP  
client.

-Doug

---

From: "Chris Lewis" <[email-masked]>
Date: Sun, 18 Jan 2009 20:34:53 -0500

> However, I wouldn't mind discussion about /how/ to get real data....

It seems to me to be part of a process.  A "threat model" comes up, and
we remember that part of the discussion should be to figure out how
likely/possible/extant they really are.

Indeed, if you don't think a threat model is realistic, it's kinda hard
to prove that it isn't - proving a negative.

In the discussion at hand, I wasn't able to give concise quantitative
numbers.  Because they are, to a great degree, unknowable.  What I did
instead was to show, by inference/within orders of magnitude, that i
wasn't happening with _huge_ numbers of "supposedly dangerous actions"
(rejecting).
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: Steve Atkins <[email-masked]>
Date: Sun, 18 Jan 2009 18:37:33 -0800

> to prove that it isn't - proving a negative.

Also, part of the process is either the deployment of the supposed
countermeasure and watching the result, or construction of a plausible
theoretical model, designed by thoughtful, well-informed people, of the
same.

Measuring how well some countermeasure works on traffic monitored
_today_, before widespread adoption of the countermeasure and
changes by other parties on the network in response to that deployment
is often going to be entirely worthless.

In those cases, the only way to measure (as opposed to predict) the
efficacy of a countermeasure is fairly large scale deployment.

> wasn't happening with _huge_ numbers of "supposedly dangerous actions"

Cheers,
   Steve