Subject: Re: [Asrg] where the message originated

From: Gordon Peterson <[email-masked]>
Date: Tue, 13 Jan 2009 18:57:54 -0600

> Ah, gotcha.  I agree that silently swallowing the message might have
spared B a possible infection, but I'm reluctant to blame A's MX for
this: it didn't originate, accept or transfer the malware-laden message.
I think responsibility lies with M, and/or with whatever system passed
the message to M (presuming the message didn't originated on M).

Well, regardless of who is pointing the finger at who, the fact remains 
that:

   1) an infected E-mail is being passed on to someone who quite likely 
had NOTHING to do with sending it, nor did they probably have any 
control over the system(s) that did;

   2) the extra bandwidth was wasted by sending mail to someone who 
didn't want it, and who can't do anything about it;

   3) the resulting infected E-mail is now coming from a computer whose 
IP address is likely in the (actual final) recipient's "trusted" zone, 
whereas the original actual origin system might well not be.

> While this is a less-than-desirable outcome, I think it may be the best
we can do -- we can't block something before we see it, we can only do
our best to reject outright spam/malware/junk traffic at the earliest
available opportunity.  (One of the ways I've expressed this elsewhere
is that the best place to stop spam is as near its source as possible.
The farther it gets from the origin, the trickier detecting it gets,
and the greater the possible consequences.)

Actually, I believe that that philosophy is incorrect.

First of all, ultimately the ONLY authority which TRULY determines FOR A 
FACT whether a given piece of e-mail is unwanted or not is the final 
recipient.

Other people enroute might conclude that something PROBABLY is spam, but 
they cannot be sure.

I understand the desirability of stopping spam and viruses as close to 
the sender as possible, WHERE THAT IS POSSIBLE, but I suggest that it 
often is not.

ULTIMATELY, the recipient of the message (i.e. the addressee, not 
necessarily the place where it ultimately gets bounced to) has a KEY 
piece of information that MTAs enroute do not have:  the knowledge of 
the (prior?) trust relationship that exists between the addressee and 
the purported sender.

And, let me point out, this even varies WITHIN a single addressee:  I 
have more than one E-mail address, and use them for different purposes. 
  An E-mail message arriving at the "wrong" address might indicate that 
it is unwanted, EVEN IF THE SAME SENDER MIGHT BE EXPECTED TO MAIL TO ME 
AT A DIFFERENT ONE OF MY ADDRESSES.

This is part of why I still believe that a finely-grained "permissions 
list", based on the recipient/orignator e-nmail address pair, AND having 
access by then to the WHOLE message, can do a SIGNIFICANTLY better job 
of accurately performing the spam/nospam triage than ANY more 
generalized SMTP-level MTA could ever hope to do enroute.

As I have pointed out before, if I get an e-mail containing an 
executable (a control panel applet, a hard .EXE file, even a piece of 
Java script or Active X enclosure or whatever) from someone who (a) I do 
not know, or (b) who I might know but who I do not expect to send me 
something like THAT, then it's perfectly reasonable to treat that e-mail 
as at least HIGHLY suspect.

Now, there ARE people I correspond with who I might EXPECT to send me 
stuff like that, or at least SOME of those sorts of things;  obviously 
those should come through to me fine.  But there may be OTHER things I 
would expect to see in mail from those trusted and familiar 
correspondents;  for example, their familiar sig that they put on the 
e-mails they send out to me, or the masthead I normally see on their 
e-newsletter, or whatever.  Even maybe the familiar way they address me 
in the accompanying message.

Note that this is not unlike the way most of us actually handle "spam 
triage" in our inboxes now:  we look to see mail coming from unfamiliar 
senders, or unfamiliar subjects, or for that matter common spam-type 
subject lines.  The key thing is that mail coming from an address we 
recognize might encourage us to look more closely at mail which we might 
not give a second thought to deleting if it came from someone unfamiliar.

I personally think that mail containing attachments (of any type) or 
HTML (scripting, ActiveX, misrepresentable links, etc) coming from 
people I don't know and trust are primary examples of mail that stands a 
strong probability of being unwanted (or at the very least, untrusted).

Now, I would agree that e-mail containing certain types of things can 
probably be axed at a very early point in the delivery process (an 
example would be mail containing a recognized virus content, especially 
a virus known to falsify return addresses...!)

BUT it is also VERY frustrating when, for example, I find that an 
absolutely legitimate e-mail message I have sent out gets bounced back 
to me by someone enroute, containing something like:

[quote]

Diagnostic-Code: smtp; 554 The message was rejected because it contains
     prohibited virus or spam content (in reply to end of DATA command)

[end quote]

...and particularly when the message contains neither virus nor spam, 
and where I have not the slightest idea of who I would need to try to 
contact (and where, or how) to solve the non-delivery problem.

And I presume that the addressee I was sending the mail to has similarly 
no idea of who they would need to complain to about our mail being 
intercepted and waylaid enroute.

Given the choice, I like the simplicity and correctability of people 
enroute delivering the mail as requested, and having the recipient's 
mail client (given the additional knowledge that only it has!) doing the 
BETTER possible job of deciding how to perform the triage.

-- 

Gordon Peterson II
http://personal.terabites.com
1977-2007:  Thirty year anniversary of local area networking
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: "Chris Lewis" <[email-masked]>
Date: Tue, 13 Jan 2009 20:31:14 -0500

>      prohibited virus or spam content (in reply to end of DATA command)

Then most of your concern should evaporate if the message said this:

This is the mail delivery agent at <redacted>
I was not able to deliver your message to the following addresses.

<[email-masked]>:
<[email-masked]>:
47.248.0.48 failed after I sent the message.
Remote host said: 552 Blocked by filters. If in error, please forward
the Session ID: 1231875923.136530000117@ecarhea1) to <redacted>@nortel.com
ONLY (#5.7.1)

Instead.  [I redacted the userids because that's a _real_ reject message.]

> contact (and where, or how) to solve the non-delivery problem.

See above.

> intercepted and waylaid enroute.

In our case, you'd presume incorrectly.

> BETTER possible job of deciding how to perform the triage.

It's been our experience that the recipients all too often lack the
experience and technical knowledge to do the triage in the most
dangerous emails.  By far the majority of our users want _us_ to stop
the crap.
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: der Mouse <[email-masked]>
Date: Tue, 13 Jan 2009 20:34:15 -0500 (EST)

> Well, regardless of who is pointing the finger at who, the fact
> remains that:

> control over the system(s) that did;

True in the scenario outlined.  But there is no way for the host
issuing the SMTP-level reject to know, in general, that that is the
case; whether a bounce to anyone is generated is up to the SMTP
client's software.  (Direct-to-MX spamware, for example, generally does
not generate bounces in reaction to rejections.)

Furthermore, even the best malware detection FPs at least occasionally.
If my mail to my friend produces a FP, the _last_ thing I want is for
it to silently vanish.  (Furthermore, the presence of malware does not
necessarily mean the mail is unwanted or shouldn't be delivered; I have
no trouble imagining researchers mailing samples to one another.  Yes,
they _can_ encrypt them or some such, but I see no a priori reason they
should have to.)

> 2) [...]

> 3) [...]

> First of all, ultimately the ONLY authority which TRULY determines
> FOR A FACT whether a given piece of e-mail is unwanted or not is the
> final recipient.

If there is one.  A lot of spam, and probably a nontrivial amount of
malware-bearing email, has no existent addresses anywhere in the
envelope (often, not in the headers either).  Who is the "final
recipient" of such a message?

> triage" in our inboxes now:  we look to see mail coming from
> unfamiliar senders, or unfamiliar subjects, or for that matter common
> spam-type subject lines.

Who's this "we"?  That's certainly not how I triage my email; the first
thing I look at for most of the mail that reaches my mailbox is the
beginning of the body.  At least a moderate fraction of my mail I never
read the Subject: or From: of at all.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		[email-masked]
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: Rich Kulawiec <[email-masked]>
Date: Tue, 13 Jan 2009 21:25:10 -0500

I'm only going to tackle two of the many points contained therein.

> FACT whether a given piece of e-mail is unwanted or not is the final  

No, not really.  There are multiple authorities which may determine whether
a given message is accepted or not.  For example: it's possible that network
engineers have implemented the Spamhaus DROP list in the border routers of
a site, that systems engineers have used the NJBAL DNSBL in a mail
system's configuration, and that an end user has used a local filter
such as Spambouncer (which does not bounce, by the way).  Thus, the
logical AND of all three permissions is required before a message will
reach a user.

This is as it should be: network engineers are probably best positioned
to use effective mechanisms like the DROP list -- doubly so since best
usage of it encompasses more than just SMTP.  Morever, network engineers,
charged with the responsibility for implementing a comprehensive network
defense policy, may -- correctly in my opinion -- conclude that *regardless*
of what any end user wants, permitting traffic to/from networks on the DROP
list is a very bad idea.  Similar considerations apply to systems admins
and end users.

End users have been known to argue that this is a Bad Thing, because
they cannot get mail they want.  I counter-argue that deliberately
allowing them to receive mail from J. Random Well-Known spammer is
every bit as dangerous to them as deliberately allowing them to
receive the Outlook virus-of-the-day, or allowing known-malicious
packets to reach the network interface on their system.  End users
counter-counter-argue that this is fascist and draconian and etc.,
but usually decline the "opportunity" to be moved outside the
effective network perimeter and receive ALL mail, sans any spam
or malware filtering, and to be exposed to ALL network traffic.

And so on, endlessly goes the policy debate.  But my point is that
in most environments, there are multiple "layers", if you will, of
accept/reject decision-making going on, and while some of those 
layers may be based on criteria like "wanted", others may not be.
None of this is a problem to the rest of us -- nobody is obligated to
accept anyone's SMTP traffic -- as long as the accept/reject decision
is clearly communicated to the outside world so that can all tell what
the heck is going on.

> contact (and where, or how) to solve the non-delivery problem.

It's been a best practice for a long time to arrange for every rejection
message to contain a human-readable string that gives some clue as to what
the problem was AND provides a means of resolving the problem, should
there be a mistake.  (This is in addition to the mandatory "postmaster"
address and the not-quite-so-mandatory-but-still-a-darn-good-idea
"abuse" address.)  Part of the problem here lies with broken mail
systems that either (a) mangle these strings or (b) substitute their
own, neatly undercutting the process.  Another part lies with end users,
who often cannot or will not take the time to read the rejection message,
or won't follow the instructions in it, or who presume that the
most likely possibility is not, perhaps, a history of spam from the
same mail system, or an errant filter, or a typo, but a personal
vendetta directed solely at them for no particular reason. [1]
And of course, most of the problem lies with mail systems that
don't even try to do this, or whose operators blithely presume
that their FP rate is 0%. [2]

Incidentally, consider how much worse this problem would get if
-- instead of returning a possibly-cryptic error message lacking
a recourse -- the recipient system simply discarded your message.

---Rsk

[1] See the Truthout & AOL debacle for an example, and note
the following for guidance in such situations:

	Vir: "Londo, they could have killed me!"
	Londo: "Nonsense, you are not important enough to kill."

[2] Maybe it's just me, but it seems that those using appliances
are particularly prone to this.  "But we paid a lot of money for it,
it can't be wrong", someone actually wrote to me.
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: "Chris Lewis" <[email-masked]>
Date: Wed, 14 Jan 2009 00:26:31 -0500

> FACT whether a given piece of e-mail is unwanted or not is the final 

You're being awfully absolute in that statement.

RSK has outlined entities (other than the final recipient) who do.  That
of course you'd counter with "but they shouldn't".

So, I'm going to take a different tack - outlining why many should.  And
in some cases legally must.

As a corporate, our email infrastructure is for our business purposes.
While we do permit reasonable personal use, exposing our infrastructure,
IP or employees to risk of malicious content is simply not on.  It's our
infrastructure, for us, we need to protect it.

For any medium-large infrastructure, they can't _afford_ to let viruses
in.  Simply from the perspective of self-preservation and integrity of
their own networks.

Secondly, at least in a business context, there are laws about workplace
environment that _require_ us to filter certain things.

> Other people enroute might conclude that something PROBABLY is spam, but 

There are many cases where "people enroute" can be absolutely certain
it's spam.  Apropos my other posting: anything saying "HELO
list.mediresource.com" (except possibly from 209.82.15.228, it's other
SPF-permitted IP helos as tempmail.mediresource.com) is spam.  Their SPF
says so, and all of the emails contain bogus DKIM signatures.  We _know_
it's a bot.

There are many times you can tell, absolutely with no FPs, that an email
is sent by a BOT.

Certainly, if you're a, say, spam or virus researcher, you might want to
get your email flow raw.  Fine.  Just don't expect a commodity ISP to
accommodate you at commodity prices.  They simply can't afford the risk
to their infrastructure.
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: der Mouse <[email-masked]>
Date: Wed, 14 Jan 2009 03:52:30 -0500 (EST)

> You're being awfully absolute in that statement.

And in the sense of "unwanted" that I think its author meant, there's
nothing wrong with that.  I think that "unwanted" is something like
"the human behind the destination address would prefer to receive the
mail".  You - and others - seem to be using values of "unwanted" that
include "I don't care who would like to receive it, we don't want to
carry it".

> Apropos my other posting: anything saying "HELO
> list.mediresource.com" (except possibly from 209.82.15.228, it's
> other SPF-permitted IP helos as tempmail.mediresource.com) is spam.

Um, no, not if it's not bulk.  Want me to telnet to your MX host and
generate an example manually? :-)

> Certainly, if you're a, say, spam or virus researcher, you might want
> to get your email flow raw.  Fine.  Just don't expect a commodity ISP
> to accommodate you at commodity prices.  They simply can't afford the
> risk to their infrastructure.

Oh, nonsense.  If that constitutes any risk to their infrastructure,
they *urgently* need to fix the bugs in question anyway!  Their
infrastructure has no need to do anything with executable - or any
other - content in the mail, except storing it for delivery to the
customer on request.  Unless they're trying to do content-based
filtering, of course, but if their content filtering setup is
vulnerable to infection from malware-bearing email, it is
catastrophically broken already; it's _job_ is to deal with such mail.

The only way I can see any risk to their infrastructure is if their
staff use their own services for mail, read mail with MUAs and OSes
vulnerable to infestation by email-borne malware, their infrastructure
hosts run similarly vulnerable OSes, and their network design permits
their staff mail-reading hosts to infect their infrastructure.  While
the first of these is likely and the second is plausible, the rest are
nobody's fault but their own.  Any risk to their infrastructure is
created by their boneheaded design and actions far more than it is by
lack of filtering on independent customer mail streams.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: Seth <[email-masked]>
Date: Wed, 14 Jan 2009 09:19:48 -0500 (EST)

> You're being awfully absolute in that statement.

Wanting is something that anybody can do.

The only authority who can truly determine if the final recipient
wants the mail is the final recipient.  And that opinion is subjet to
change over time, even for the same piece of mail.  (User gets an ad
to take a friend to a particular movie for free, does so, and the
movie is so awful he wishes he hadn't received the ad.)

The only authority who can determine whether I want the final
recipient to receive the mail is me.

The only authority who can determine whether the final recipient's
employer wants the final recipient to recieve the mail is the final
recipient's employer.

Etc.

> Certainly, if you're a, say, spam or virus researcher, you might
> want to get your email flow raw.  Fine.  Just don't expect a
> commodity ISP to accommodate you at commodity prices.  They simply
> can't afford the risk to their infrastructure.

Some will; what's the risk to a BSD infrastructure of letting
customers receive Windoze viruses?  (Sure, they have to be careful
about outgoing mail, but that's true anyway because there are plenty
of non-email ways for lusers to become infected.)

Seth

---

From: "Chris Lewis" <[email-masked]>
Date: Wed, 14 Jan 2009 10:54:02 -0500

> generate an example manually? :-)

Except for mice in the works ;-)

It may not be bulk, but forgery in the face of the domain owner
asserting otherwise is still undesirable.

> they *urgently* need to fix the bugs in question anyway!

Oh, nonsense on your nonsense! ;-)

Perhaps I should have said "network" instead of "infrastructure".  But
the fact remains that an infection on one recipient's machine has
consequences far beyond _that_ machine alone.

The components of an infrastructure do not need to be infected to be
damaged by infections.  Surely you remember slammer?  A handful of
infected desktops took out out entire networks without infecting
anything further.  Simply by bandwidth exhaustion.

Then of course there's the potential reputation loss if something
manages to spam outwards through your MTAs.

Nor does choice of O/S barricade you more than quantitatively.  As
secure as *BSD (for example, for you Seth ;-), it's not immune to going
awry, especially in the face of naive users poking at things they
shouldn't be.
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg