Subject: Re: [Asrg] Re: Asrg Digest, DNSBL BCP v.2.0

From: <[email-masked]>
Date: Sun, 04 Mar 2007 13:18:04 -0600

>>> so I assume that the infected machine was using the companies own
>smtp server as a smarthost and that is why the entire company was
>inconvinienced. 

>> No. It was sending (apparently) using its own SMTP sending engine,
but behind the (single) NAT router and therefore from the 
Internet
side was indistinguishable (by IP address, anyhow) from 
any other
E-mails coming from within the entire company, from any of 
their
inhouse outgoing mail servers.

> Simple question... *WHY WAS THE ROUTER/GATEWAY NOT BLOCKING PORT 25 TO/FROM ALL MACHINES EXCEPT AUTHORIZED INTERNAL MTAS* ??? If your
client had taken that one simple step, none of this 
would've happened.

Several issues there.

First, they have at least three or four internal machines 
(out of only about 15) running mail servers. (These 
servers were basically used as a speed buffer/queue for 
outgoing mail only).

Second, their applications (running on many of their 
client machines) can be configured (and recently were, as 
a workaround) to send directly to outside mail servers.

Third, the primary machine involved with their infection 
was in fact one of the machines running not just a mail 
server, but a critical app which does legitimately send 
E-mails as a key part of its job.

Fourth, their NAT router/firewall was provided, installed, 
and maintained by their telephone company who had 
basically no knowledge or understanding of the company's 
internal IT systems. (This is a very common situation, 
both for home users and small businesses). Phone companies 
typically install these things, and walk away from them.

Obviously, I'm going to have to be more involved with 
actively administering their router, in the future.

Gordon Peterson
http://personal.terabites.com
1977-2007 Thirty year anniversary of local area networking

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Matt Sergeant <[email-masked]>
Date: Sun, 4 Mar 2007 15:17:55 -0500

On 4-Mar-07, at 2:18 PM, <[email-masked]> <[email-masked]>  
wrote:

>> Simple question... *WHY WAS THE ROUTER/GATEWAY NOT BLOCKING PORT  
>> 25 TO/FROM ALL MACHINES EXCEPT AUTHORIZED INTERNAL MTAS* ??? If your
> client had taken that one simple step, none of this would've happened.
>
> Several issues there.
>
> First, they have at least three or four internal machines (out of  
> only about 15) running mail servers. (These servers were basically  
> used as a speed buffer/queue for outgoing mail only).

Jeez - how much email does this 15 person company send??? A  
reasonable mail server can handle a million mails an hour - just how  
much "speed" do they need?

> Third, the primary machine involved with their infection was in  
> fact one of the machines running not just a mail server, but a  
> critical app which does legitimately send E-mails as a key part of  
> its job.

So lets get this straight - the mail server was being used as a  
desktop machine? I see no other way it could have been infected with  
a spam trojan than someone happened to be using it as a desktop.

I'm sorry this happened to the company you administer, but clearly it  
has taught you some really important lessons about corporate network  
security that you can apply to future contracts. Frankly you should  
probably be glad they got CBL listed.

Matt.

______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
______________________________________________________________________

---

From: Douglas Otis <[email-masked]>
Date: Sun, 4 Mar 2007 17:20:58 -0800

On Mar 4, 2007, at 12:17 PM, Matt Sergeant wrote:

> reasonable mail server can handle a million mails an hour - just  
> how much "speed" do they need?

While blocking port 25 is a good idea, this does not assure  
protection.  Messages can still be mailed from infected systems.   
Most systems are configured to automatically forward and authenticate  
messages originating within the system.  This automation affords bad  
actors greater cover, as stopping such outbound servers results in  
greater collateral damage.  Bad actors don't need to infect systems  
be behind a NAT for that trick to work.

Although blocking port 25 frustrates proponents of the end-to-end  
ideal, imposing external boundaries has become standard fair.   
Although when XP was first announced, Microsoft made a statement XP  
will not requires a firewall.  As it is now, XP can not be installed  
without being behind a firewall or a NAT.  The meantime-before- 
infection does not afford time to install updates.  As services such  
as Teredo and UPNP become common, compliant SOHO routers may usher in  
a era where NAT's Stateful Packet Inspection must understand new,  
albeit often undocumented, extensions.

Perhaps in a few years there will be a forklift changeover to IPv6  
when NATs and firewall automation goes awry, where network and memory  
manufactures rejoice.  When that happens, this will likely thwart IP  
address scrutiny, or reverse lookup schemes.  IPv6 may mean public  
"anonymous" messages can not be accepted without first confirming a  
CA certificate.  It is obvious having a registered domain will not  
represent an impediment to abuse.  The cost of sending an message may  
someday be a CA cert.  While something like DKIM offers a means for  
domains to assert their certificates, some type of accreditation  
analogous to a CA is still be required.

A possible replay issue may also thwart using transmitter independent  
domains as a basis for acceptance, although authorizations schemes  
may provide possible solutions.  These authorizations should be by  
name, where iterative IP address lists should be avoided due to  
related DDoS threats.

-Doug

---

From: "Walter Dnes" <[email-masked]>
Date: Sun, 4 Mar 2007 22:22:02 -0500

On Sun, Mar 04, 2007 at 01:18:04PM -0600, [email-masked] wrote

> >Simple question... *WHY WAS THE ROUTER/GATEWAY NOT BLOCKING PORT 25 
> >TO/FROM ALL MACHINES EXCEPT AUTHORIZED INTERNAL MTAS* ??? If your

  3 or 4 machines to handle outbound email produced by 11 or 12 client
machines doesn't sound very efficient.  A couple of BSD or linux
machines (without a GUI installed) should be able to handle the load.
One should be sufficient, the second one would be there as a spare in
case of hardware failure on the first.

> Second, their applications (running on many of their client machines)
> can be configured (and recently were, as a workaround) to send
> directly to outside mail servers.
> 
> Third, the primary machine involved with their infection was in fact
> one of the machines running not just a mail server, but a critical
> app which does legitimately send E-mails as a key part of its job.

  I cannot adequately express my opinion about the second and third
points, without breaking "forum-decorum".

> Fourth, their NAT router/firewall was provided, installed, and
> maintained by their telephone company who had basically no knowledge
> or understanding of the company's internal IT systems. (This is a very
> common situation, both for home users and small businesses). Phone
> companies typically install these things, and walk away from them.
> 
> Obviously, I'm going to have to be more involved with actively
> administering their router, in the future.

  OK, here are a few suggestions.  Two or three linux and/or BSD
machines with MTA software, but otherwise minimal OS (no GUI), should
be the only machines allowed to talk to the outside world on port 25.
No ifs-ands-ors-buts.  Enforce this rule at the router.

  When I say "MTA", I'm including a malware scanner for *OUTBOUND*
email, so that even if one of the client machines gets hit by "swen",
which piggybacks on the legitimate MTA, executables and other malware
is still not sent out.  This addresses your concern about the fact that
sooner or later, one of the client machines *WILL* get hit by malware.

  The *nix machines will only be running an MTA, no client software.
The client machines will *NOT* be running an MTA; they'll send their
email via the 2 or 3 *nix machines.

  Get into BOFH mode and block all outside access by the client
machines, see what breaks, and allow only what is necessary for the job.

-- 
Walter Dnes <[email-masked]> In linux /sbin/init is Job #1

---

From: <[email-masked]>
Date: Sun, 04 Mar 2007 23:56:30 -0600

On Sun, 4 Mar 2007 15:17:55 -0500
  Matt Sergeant <[email-masked]> wrote:
> On 4-Mar-07, at 2:18 PM, <[email-masked]> 
><[email-masked]>  wrote:
> 
> Jeez - how much email does this 15 person company 
>send??? A  reasonable mail server can handle a million 
>mails an hour - just how  much "speed" do they need?

It's less an issue of speed than it is of simply 
convenience and auditability.  The mail server software 
they use is simple freeware "EMWAC", not fancy but it 
suits their needs well.  They probably send between 
600-1000 E-mails per day, as far as their automatically 
system-generated ones go.  Add to that whatever E-mails 
they personally send.

>>was in  
> So lets get this straight - the mail server was being 
>used as a  desktop machine? I see no other way it could 
>have been infected with  a spam trojan than someone 
>happened to be using it as a desktop.

That's right.  The EMWAC mail server is an additional 
function on a machine they already have and are using 
(occasionally) as a desktop.  That particular machine is 
also running an incoming mail processing bot I wrote for 
them, which retrieves incoming mail from an external 
mailbox, tracks outgoing E-mails and fax-by-mail messages, 
scheduling and reissuing them as necessary, routing 
various classes of incoming messages, reports on message 
deliveries (and non-delivery), eventually deletes 
delivered mail from the retry-holding queues, etc etc.

> I'm sorry this happened to the company you administer, 
>but clearly it  has taught you some really important 
>lessons about corporate network  security that you can 
>apply to future contracts. Frankly you should  probably 
>be glad they got CBL listed.

Well, it certainly HAS been a learning experience, but not 
so much because I don't understand network security.  ;-) 
 I certainly am even more convinced than before that 
wholesale IP address blocking is a terrible solution, 
because of the unacceptably blunt nature of the bludgeon.

The other issue, of course, is that IP address blocking is 
(again) REACTIVE, and only takes effect AFTER some amount 
of worm or spam traffic has been observed.  Far better to 
instead not deliver the FIRST of the junk.

Also, and I should have replied to that other message 
earlier, but on the same lines, there was a question about 
when an RBL blacklisting should expire.  I would think a 
listing should expire some number of hours after the last 
such bogus message was seen coming from that host.... 
rather than having to be removed manually.  (And the reset 
delay could, for example, be increaed each time a 
subsequent infected message from that address (or, better, 
machine) is seen.

But again, NAT makes blocking a NAT router a terrible idea 
(and worse, depending on how many machines are downstream 
from that router).

Gordon Peterson
http://personal.terabites.com
1977-2007  Thirty year anniversary of local area 
networking

---

From: Daniel Feenberg <[email-masked]>
Date: Mon, 5 Mar 2007 07:05:04 -0500 (EST)

On Sun, 4 Mar 2007, Douglas Otis wrote:

<snip>>
>
> Perhaps in a few years there will be a forklift changeover to IPv6 when NATs 
> and firewall automation goes awry, where network and memory manufactures 
> rejoice.  When that happens, this will likely thwart IP address scrutiny, or 
> reverse lookup schemes.  IPv6 may mean public "anonymous" messages can not be 
> accepted without first confirming a CA certificate.  It is obvious having a 
> registered domain will not represent an impediment to abuse.  The cost of 
> sending an message may someday be a CA cert.  While something like DKIM 
> offers a means for domains to assert their certificates, some type of 
> accreditation analogous to a CA is still be required.
>

Is smtp mail ever likely to switch to IPV6? I don't anticipate ever 
accepting mail from IPv6 addresses on our mailserver, simply because there 
are too many addresses for Spamhaus to keep track of. Given the difficulty
IPV6 is having already I would guess that even if IPv6 became very 
widespread, it would never even gain a foothold for MTA senders.

I understand that in Japan IPV6 is well established, does anyone know if 
Japanese mail servers ever use it? I understand that there are plans in 
the US Government to require agencies to use IPv6. Does anyone know if 
those plans anticipate forbidding agencies from sending/receiving mail at 
IPv4 addresses?

As (if?) IPv6 becomes more widespread, the operator of an MTA may have the 
choice of keeping his IPv4 address, or getting a certificate as this 
poster suggests. Since he already has the address, and the certificate 
will be complicated and expensive to acquire (and of limited use, since 
few receivers will recognize any particular certificate), I think the 
decision will be to do nothing. New entrants will either manage to obtain 
a single IPv4 address for the MTA, or use a smarthost. I don't anticipate 
addresses will ever be in such shortage that it will be harder to get 
*one* than any imaginable certificate.

An alternative scenario is for Spamhaus to start listing IPv6 addresses, 
but in blocks of /48. But that hardly seems likely. I think MTA operators 
would rather just avoid IPv6 altogether.

Daniel Feenberg

---

From: der Mouse <[email-masked]>
Date: Mon, 5 Mar 2007 11:16:14 -0500 (EST)

> Is smtp mail ever likely to switch to IPV6?

Switch?  I don't know.  Certainly the IPv4 address space can stand
holding everyone's mailserver a lot easier than it can stand holding
all of everyone's hosts. :-)

But I do know that my mailserver was accepting mail over v6 until my v6
connectivity broke a year ago (my upstream's upstream went poof, and,
while they and I have renumbered, they haven't yet found another
upstream - I suspect it's just low priority for them, being in North
America and all).  And by "accepting mail" I mean "actually receiving
mail", not "being prepared to receive mail".

/~\ The ASCII				der Mouse
\ / Ribbon Campaign
 X  Against HTML	       [email-masked]
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

---

From: John Levine <[email-masked]>
Date: 5 Mar 2007 16:47:44 -0000

>Is smtp mail ever likely to switch to IPV6?

Eventually, sure.  Some time in the next decade IANA will allocate the
last available /8 to the RIRs, and IPv4 space will get a lot more
expensive.

We'll probably start seeing v6-only hosts in Asia first, and people
will upgrade depending on the pain level of being unreachable from
those places.  Given that most Americans are barely aware that Canada
exists, much less countries in Asia, I imagine it'll take a while.  I
also expect formal and informal v6 to v4 gateways to spring up, with
all of the spammy excitement of any third party gateway.

Most network equipment shipped these days can handle v6, so when you
finally switch over, it's just going to be a config change.

R's,
John

---

From: "Chris Lewis" <[email-masked]>
Date: Mon, 05 Mar 2007 12:04:59 -0500

[email-masked] wrote:

> Also, and I should have replied to that other message earlier, but on
> the same lines, there was a question about when an RBL blacklisting
> should expire.  I would think a listing should expire some number of
> hours after the last such bogus message was seen coming from that
> host.... rather than having to be removed manually.  (And the reset
> delay could, for example, be increaed each time a subsequent infected
> message from that address (or, better, machine) is seen.

The CBL expires entries.  Fairly quickly at that, but not quick enough
in the case of critical email.  But you didn't notice that you could
remove an IP from the CBL on demand, did you?

The CBL is, as far as I can tell, in full compliance with the BCP and
then some.  If you had followed the instructions on their web page, you
could have solved the problem completely in an hour or two, not many days.

Now, if it had been other less-BCP-compliant DNSBLs, such as BLARS or
SPEWS, getting out of them would be very difficult, and in fact
impossible in the case of BLARS.

Hence, the value of the BCP.

> But again, NAT makes blocking a NAT router a terrible idea (and worse,
> depending on how many machines are downstream from that router).

Yes, it's a terrible idea.  But once you identify that an IP is sending
malicious content, it's an even more terrible idea to continue to allow
it to do so.  And if you _do_ continue to allow it, even if you do
manage to swat the malicious content itself letting the good content
through, there's no incentive to fix the problem.

There are plenty of providers out there who will do _nothing_ to solve
malicious emitters unless you give them some pain.

Yes, DNSBLs like the CBL are reactive.  But not all DNSBLs are reactive.
 The PBL isn't for example.

Yes, it would be nice to prevent the malicious content being sent in the
first place - that's in your (or your customer's court), because they
didn't.  If the proactive measures fail, which they did, then, reactive
is what's left.

---

From: Douglas Otis <[email-masked]>
Date: Mon, 5 Mar 2007 13:36:31 -0800

On Mar 5, 2007, at 8:47 AM, John Levine wrote:

> Eventually, sure.  Some time in the next decade IANA will allocate  
> the last available /8 to the RIRs, and IPv4 space will get a lot  
> more expensive.
>
> those places.  Given that most Americans are barely aware that  
> Canada exists, much less countries in Asia, I imagine it'll take a  
> while.  I also expect formal and informal v6 to v4 gateways to  
> spring up, with all of the spammy excitement of any third party  
> gateway.

Although IPv6 is commonly used in Japan, it is typically used for  
internal services.  Indeed IPv6 to IPv4 gateways will become  
problematic, as will any other large NAT.  Looking to see the  
durations of a route offers clues.  Identifying those managing the  
routing helps in recognizing different paradigms and what is involved.

> Most network equipment shipped these days can handle v6, so when  
> you finally switch over, it's just going to be a config change.

Defensive practices valid for IPv4 are unlikely to remain a viable  
strategy within a predominate IPv6 environment.  Many security  
related strategies are likely to need rethinking.  IPv6 will not  
solve black-hole listing problems, or remove problems associated with  
NATs, unless dependence upon the IP address is replaced with  
dependence upon a CA Cert perhaps.

-Doug