Subject: [Asrg] Some data on the validity of MAIL FROM addresses

From: Kee Hinckley <[email-masked]>
Date: Sun, 18 May 2003 03:34:14 -0400

Vernon has regularly made the claim that a significant proportion of 
spam messages have valid MAIL FROM's.  That means that bounces will 
go the the spammer.  This has significant ramifications for C/R 
systems (especially auto-respond ones) since it means that should 
they have to, spammers could respond to challenges.

To test this theory, I took a day's worth of bounce logs from 
somewhere.com (2003-05-15).  These should be fairly normal logs. 
There's been a bit of an upswing from a recent virus attack, but 
otherwise these are pretty normal bounce logs for somewhere.com. 
These are for addresses that do not, and have never, existed. 
Because they got on the spammer's lists primarily because someone 
entered the address on a web site, they get a mix of "true" spam and 
just standard bulk mail.  However if they bulkmailers are doing their 
job, those addresses should be removed fairly quickly.  If they 
aren't removing on bounces--then they look and smell a lot like 
spammers.

Known oddities in the data:

862 messages to [email-masked] and variations.  These tend 
to run about 1/3 viruses, 1/3 real messages and 1/3 spam.  That set 
has 533 distinct MAIL FROM addresses.

12340 messages from [email-masked] to [email-masked]. 
(Misconfigured Axis video cameras.)

Since all I'm counting here are unique MAIL FROM addresses, neither 
of these should have a huge impact.

I ran a program which took each MAIL FROM address, parsed out the 
domain portion, looked up the MX record, and then connected to the 
SMTP port of the lowest numbered MX server.  I did a
	HELO somewhere.com
	MAIL FROM <[email-masked]>
	RCPT TO <appropriate-address>
	QUIT
Note that a few sites bounced me at the HELO prompt (didn't like that 
I was on DSL, or that my name was somewhere.com).  A few bounced at 
the MAIL FROM (didn't like somewhere.com--and one claimed that + 
wasn't a legal email character).  But the number of either of those 
was pretty low (less than half a dozen).  I'll do a better job of 
recording those separately in the future.

There were 39595 entries in the log, with 34404 distinct SMTP sessions.
There were 11559 unique MAIL FROM addresses.

+---------+-------+------------+
| errcode | total | percentage |
+---------+-------+------------+
|       0 |    99 |       0.86 |	???
|     250 |  5796 |      50.14 |
|     450 |     6 |       0.05 |
|     451 |    12 |       0.10 |
|     452 |     8 |       0.07 |
|     473 |     4 |       0.03 |
|     500 |     1 |       0.01 |
|     501 |     1 |       0.01 |
|     521 |     3 |       0.03 |
|     530 |     1 |       0.01 |
|     550 |  2341 |      20.25 |
|     551 |     3 |       0.03 |
|     552 |     2 |       0.02 |
|     553 |   288 |       2.49 |
|     554 |    48 |       0.42 |
|     555 |     1 |       0.01 |
|     556 |     1 |       0.01 |
|     571 |     1 |       0.01 |
|    1001 |  1880 |      16.26 |	No MX Record
|    1003 |  1055 |       9.13 |	No SMTP Server
|    1007 |     8 |       0.07 |	Invalid Email Format
+---------+-------+------------+

In aggregate.  51% of the addresses were valid.  49% were not.
Of the ones that were not valid, 52% didn't have a reachable mail server.

Now let's see how it breaks down by domain.

Here are the top 5 domains in the MAIL FROM's.
+-------------------------+-------+
| host                    | count |
+-------------------------+-------+
| yahoo.com               |   819 |
| hotmail.com             |   714 |
| aol.com                 |   632 |
| earthlink.net           |   209 |
| msn.com                 |   161 |
+-------------------------+-------+

Let's do the same stats for each of these.  Note that I have a 1-2% 
"No SMTP Server" rate.  This could mean that they were rate limiting 
my queries.  More likely it's do the the very short timeout I put on 
doing the query.  I'll have to adjust that in the future.

+-----------+---------+-------+------------+
| host      | errcode | total | percentage |
+-----------+---------+-------+------------+
| yahoo.com |    NULL |     1 |       0.12 |
| yahoo.com |     250 |   669 |      81.68 |
| yahoo.com |     553 |   129 |      15.75 |
| yahoo.com |    1003 |    20 |       2.44 |
+-----------+---------+-------+------------+
+-------------+---------+-------+------------+
| host        | errcode | total | percentage |
+-------------+---------+-------+------------+
| hotmail.com |    NULL |     1 |       0.14 |
| hotmail.com |     250 |   111 |      15.55 |
| hotmail.com |     550 |   602 |      84.31 |
+-------------+---------+-------+------------+
+---------+---------+-------+------------+
| host    | errcode | total | percentage |
+---------+---------+-------+------------+
| aol.com |       0 |    10 |       1.58 |
| aol.com |     250 |   581 |      91.93 |
| aol.com |     550 |    10 |       1.58 |
| aol.com |    1003 |    31 |       4.91 |
+---------+---------+-------+------------+
+---------------+---------+-------+------------+
| host          | errcode | total | percentage |
+---------------+---------+-------+------------+
| earthlink.net |     250 |    43 |      20.57 |
| earthlink.net |     550 |   149 |      71.29 |
| earthlink.net |     554 |    14 |       6.70 |
| earthlink.net |    1003 |     3 |       1.44 |
+---------------+---------+-------+------------+
+---------+---------+-------+------------+
| host    | errcode | total | percentage |
+---------+---------+-------+------------+
| msn.com |    NULL |     1 |       0.62 |
| msn.com |     250 |    62 |      38.51 |
| msn.com |     550 |    97 |      60.25 |
| msn.com |    1003 |     1 |       0.62 |
+---------+---------+-------+------------+

Interesting that the results vary so much by ISP.  Yahoo accounts are 
pretty valid.  Hotmail accounts are pretty bad.  AOL is quite good. 
Earthlink has a problem.  MSN's slightly better, but still negative.

In general though, it appears that Vernon is correct.  If my sample 
is representative, a large percentage of spam is coming from real 
email addresses.

I'll be making this data (and hopefully live update's to it) 
available on the web, hopefully in the next few days.

As an addition anecdotal piece of information.  In the past month 
I've seen five separate email accounts (including two of mine) get 
Joe-jobbed in a new way.  Instead of major bounceback, they just get 
one or two.  It smells like new spam software that uses the same 
database of addresses for From that they were using for To.  The goal 
might be to get through verification filters like the above.  But 
it's also interesting to consider what havoc that might wreak on C/R 
systems.  How is someone going to react with they get a challenge for 
a message they didn't send?  I predict that if people get used to C/R 
systems they'll just click send--and the spammer's message will get 
through.

Finally, as an addendum of sorts.  Here are the unique messages 
associated with the above error codes.  I've left out 250 and 550 
ones--I'm just tracking the less common ones.  And they've been 
normalized to remove email addresses and domain names.

+---------+-------+----------------------------------------------------+
| errcode | count | substring(message,1,50)                            |
+---------+-------+----------------------------------------------------+
|       0 |    99 |                                                    |
|     250 |  5796 | recipient ok                                       |
|     450 |     3 | <EMAILADDRESS>: User unknown in local recipient ta |
|     450 |     2 | <localhost.localdomain>: Helo command rejected: Ho |
|     450 |     1 | Mailbox unavailable.                               |
|     451 |     1 | 4.0.0 Can't create transcript file ./xfh4GNNYv0581 |
|     451 |     1 | 4.3.0 error creating message, status = StatusSpool |
|     451 |     1 | 4.3.5 Error getting LDAP results in map sbcldap:   |
|     451 |     4 | <EMAILADDRESS>: Temporary lookup failure           |
|     451 |     1 | <LOCALPART> ... Recipient mailbox is full          |
|     451 |     1 | Can't connect to bisman.com - psmtp                |
|     451 |     2 | Requested action aborted: local error in processin |
|     451 |     1 | Server Error                                       |
|     452 |     2 | 4.2.1 Mailbox temporarily disabled: EMAILADDRESS   |
|     452 |     2 | 4.2.2 Mailbox full                                 |
|     452 |     2 | 4.4.5 Insufficient disk space; try again later     |
|     452 |     2 | Message for <EMAILADDRESS> would exceed mailbox qu |
|     473 |     4 | EMAILADDRESS relaying prohibited. You should authe |
|     500 |     1 | <EMAILADDRESS>: Recipient address rejected: Recipi |
|     501 |     1 | Syntax error in sender: <postmaster+AntiSpamAddres |
|     521 |     1 | This User has too many concurrents, please try aga |
|     521 |     2 | this mailbox is disabled or invalid (#5.2.1)       |
|     530 |     1 | Delivery not allowed to non-local recipient, try a |
|     550 |  2341 | unknown user                                       |
|     551 |     1 | 5.0.0 Mailbox disabled,storage space exceeded      |
|     551 |     1 | EMAILADDRESS illegal name for an account           |
|     551 |     1 | not our customer                                   |
|     552 |     1 | <EMAILADDRESS>: Recipient address rejected: Sorry, |
|     552 |     1 | Requested action aborted: exceeded storage allocat |
|     553 |     1 | 5.0.0 <EMAILADDRESS>... No such user               |
|     553 |     1 | 5.1.3 <EMAILADDRESS>... Invalid route address      |
|     553 |    17 | 5.3.0 <EMAILADDRESS>... Addressee unknown, relay=[ |
|     553 |     1 | 5.3.0 <EMAILADDRESS>... Delivery ERROR!!!User does |
|     553 |     4 | 5.3.0 <EMAILADDRESS>... No such user               |
|     553 |     6 | 5.3.0 <EMAILADDRESS>... No such user here          |
|     553 |     2 | 5.3.0 <EMAILADDRESS>... That address is not curren |
|     553 |     1 | 5.3.0 <EMAILADDRESS>... Try [email-masked] |
|     553 |     1 | 5.3.0 <EMAILADDRESS>... User LOCALPART mailbox ful |
|     553 |     3 | 5.3.0 <EMAILADDRESS>... User unknown               |
|     553 |     8 | 5.5.3 <EMAILADDRESS>... Invalid                    |
|     553 |     1 | <EMAILADDRESS>... User unknown                     |
|     553 |     2 | No mailbox here by that name, sorry (#5.7.1)       |
|     553 |     1 | RCPT TO:<EMAILADDRESS> refused                     |
|     553 |     7 | Requested action not taken: mailbox name not allow |
|     553 |   143 | VS10-RT Possible forgery or deactivated due to abu |
|     553 |    88 | sorry, that domain isn't in my list of allowed rcp |
|     553 |     1 | sorry, your envelope sender is in my badmailfrom l |
|     554 |     1 | 5.0.0 ADMIN.COM ISN'T THE DOMAIN YOU'RE LOOKING FO |
|     554 |     3 | <EMAILADDRESS>: Recipient address rejected: Access |
|     554 |     1 | <EMAILADDRESS>: Recipient address rejected: Domain |
|     554 |     9 | <EMAILADDRESS>: Recipient address rejected: Not ac |
|     554 |     1 | <EMAILADDRESS>: Recipient address rejected: Relay  |
|     554 |     5 | <EMAILADDRESS>: Relay access denied                |
|     554 |     1 | <localhost.localdomain>: Helo command rejected: Ho |
|     554 |     1 | EMAILADDRESS Mail quota exceeded                   |
|     554 |     1 | Mail for EMAILADDRESS rejected for policy reasons. |
|     554 |    21 | Quota violation for EMAILADDRESS                   |
|     554 |     1 | Relay rejected for policy reasons.                 |
|     554 |     2 | SPAM-Relay detected                                |
|     554 |     1 | recipient <EMAILADDRESS>, Transaction failed       |
|     555 |     1 | sorry, your envelope recipient is in my badrcptto  |
|     556 |     1 | invalid email address EMAILADDRESS (5.5.6)         |
|     571 |     1 | <www.somewhere.com[66.92.72.194]>: Client host rej |
|    1001 |  1880 | No MX Record                                       |
|    1003 |  1055 | No SMTP Connection                                 |
|    1007 |     8 | Bad Address Format                                 |
+---------+-------+----------------------------------------------------+

-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

I'm not sure which upsets me more: that people are so unwilling to accept
responsibility for their own actions, or that they are so eager to regulate
everyone else's.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: "Jon Kyme" <[email-masked]>
Date: Sun, 18 May 2003 10:53:23 +0100

> Earthlink has a problem.  MSN's slightly better, but still negative.

Of course yahoo will say 250 to pretty much anything.
So these addresses are "valid" in what sense exactly?

---

From: Vernon Schryver <[email-masked]>
Date: Sun, 18 May 2003 07:02:44 -0600 (MDT)

> From: Kee Hinckley <[email-masked]>

> ...

> ...

Actually, my claim differs somewhat.  It is that most spam with free
provider MAIL_FROM values is not "forged" but that the spammer can
legitimately claim to own the MAIL_FROM value even if it has since
been terminated by the ISP.  Your data is consistent with my claim as
well as with the widely believed belief that Hotmail is much quicker
to terminate spam drop-boxes than Yahoo.

Long ago AOL had a problem in providing drop-boxes to spammers.  By
many accounts, they fixed that problem about the time AOL MAIL_FROM
values largely disappeared from spam.  AOL representatives said they
fixed the problem by various mechanisms including requiring and checking
credit card numbers of all new subscribers before or during account
acctiviation.  In recent months I've noticed a lot of spam with AOL
MAIL_FROM values.  Recent AOL "free" offers trumpet not requiring a
credit card number for account activation.

Vernon Schryver    [email-masked]
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Fred Bacon <[email-masked]>
Date: 18 May 2003 12:56:42 -0400

> aren't removing on bounces--then they look and smell a lot like 

<snip>

> available on the web, hopefully in the next few days.

I nice idea, but what we really need is the script you used to analyze
your logs.  Then additional data can be collected at a variety of
locations.  

I realize that there are many on this list who find data collection to
be pointless, but Kee Hinckley has shown this to be incorrect.  Vernon
Schryver's assertions were useless (even if correct) without hard
evidence, and Kee's data is insufficient without wider deployment.

Likewise, Vernon's followup that Kee is analyzing a different statement
than Vernon asserted is a legitimate concern.  The data analysis
methodology should be publicly vetted to ensure that it is providing
meaningful and acurate data.

Paul, is it possible for the www.irtf.org/asrg website to host log
analysis tools?  This is directly applicable to the list of Work Items.

-- 
Fred Bacon <[email-masked]>
Aerodyne Research, Inc.

---

From: Yakov Shafranovich <[email-masked]>
Date: Sun, 18 May 2003 14:54:06 -0400

>they have to, spammers could respond to challenges.

I believe that we mentioned before, that if the sender's system supports 
C/R and keeps track of all outgoing messages, then it can compare the 
X-CR-Recipient: header against the list of email addresses this user send 
email to. This avoids the problem of spammers using a real email address 
for the FROM addresses. However, if the spammers themselves are operating 
the mail servers there is nothing we can do but at the last we would know 
where the email came from and we can track it down.

Yakov

---------------------------------------------------------------------------------------------------
Yakov Shafranovich / <[email-masked]>
SolidMatrix Research, a division of SolidMatrix Technologies, Inc.
---------------------------------------------------------------------------------------------------
"One who watches the wind will never sow, and one who keeps his eyes on
the clouds will never reap" (Ecclesiastes 11:4)
---------------------------------------------------------------------------------------------------

---

From: Vernon Schryver <[email-masked]>
Date: Sun, 18 May 2003 17:17:11 -0600 (MDT)

> From: Yakov Shafranovich <[email-masked]>

> where the email came from and we can track it down.

If spammers have valid return addresses, then what distinguishes challenges
of their mail or their responses from the same for anyone else?

Are you assuming that legitimate mail comes only the system named
by the sender domain?  That restriction is similar to but stronger
than the basic RMX assumption.

Are you expecting not only that distant users will respond to challenges,
but that their ISPs will modify their MTAs to maintain databases of
all addresses to which their users have sent mail?  If so, that
seems...implausible.

---

From: Yakov Shafranovich <[email-masked]>
Date: Sun, 18 May 2003 20:27:15 -0400

At 05:17 PM 5/18/2003 -0600, Vernon Schryver wrote:
>[...]

I did expect ISPs to maintain such databases for only for a limited time. 
An average challenge/response interaction if done by automated software 
only takes moments or at most minutes. If an ISP is implementing an 
automated C/R system, they might keep track of outgoing addresses just like 
they keep a whitelist and queue for incoming. However, that would be an 
optional part of the draft.

What I do expect is that users will not reply to challenges that come from 
strangers - if spammers start using someone's email address for the "FROM" 
field, it would make sense to assume that the user in question will not 
respond to thousands of incoming messages.

Yakov

---

From: Kee Hinckley <[email-masked]>
Date: Sun, 18 May 2003 20:58:51 -0400

>So these addresses are "valid" in what sense exactly?

They said "no" to 16% of the messages I queried them on.  The 
specific message they used was:

553 VS10-RT Possible forgery or deactivated due to abuse (#5.1.1)

Can you show instances in which they say yes to messages they cannot deliver?
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: Kee Hinckley <[email-masked]>
Date: Sun, 18 May 2003 21:01:58 -0400

>been terminated by the ISP.  Your data is consistent with my claim as

Understood.  One reason I chose a recent sample was to try and avoid 
missing accounts due shutdown.  Those tests were run within 24 hours 
of the time I received the email.  Of course, we have no way of 
knowing when the spammer set up their software, or how long they've 
been using that particular account.
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: Yakov Shafranovich <[email-masked]>
Date: Sun, 18 May 2003 21:55:43 -0400

At 08:58 PM 5/18/2003 -0400, Kee Hinckley wrote:

>They said "no" to 16% of the messages I queried them on.  The specific 
>message they used was:
>

Well they can say yes even though it cannot be delivered - see RFC 2821, 
section 3.3.

Ykaov

---

From: Vernon Schryver <[email-masked]>
Date: Sun, 18 May 2003 20:08:19 -0600 (MDT)

> Can you show instances in which they say yes to messages they cannot deliver?

I was skeptical of the report that Yahoo defers no-such-user responses
until the DATA command, and so I made the obvious `telnet ... 25` test. 
I tried a perfectly valid message to an address that was very unlikely
to exist.  The Rcpt_To command was answered with at 250 response.  It
was not until the end of the DATA command that I got a 5yz.

Thus, Yahoo's "VS10-RT Possible forgery or deactivated due to abuse"
can reasonably be counted as "this account was owned by the spammer"
and while the 250's must be counted as "don't know".  (I doubt you'd
want to send unsolicited mail to any address that you suspect was
forged into spam.)

It occurs to me that delaying the 5yz-unknown-user response to the
DATA command could be handy for people running body filters.

Knowledgeable people have suggested that some dictionary attack
spammers first try random, practically certain to be invalid target
names before launching into their dictionaries in order to determine
if the target SMTP server answers 250-OK for bogus addresses.

 ]]]]]]]]]]]]]]]]]]]]]]]]]]]]

] From: Kee Hinckley <[email-masked]>

] ...
] Understood.  One reason I chose a recent sample was to try and avoid 
] missing accounts due shutdown.  Those tests were run within 24 hours 
] of the time I received the email.  Of course, we have no way of 
] knowing when the spammer set up their software, or how long they've 
] been using that particular account.

Some people use (or talk about using) a Rcpt_To test on sender
addresses to detect spam.  Their false positive and negative ratios
would be interesting.

---

From: Kee Hinckley <[email-masked]>
Date: Sun, 18 May 2003 22:23:56 -0400

At 8:27 PM -0400 5/18/03, Yakov Shafranovich wrote:
>What I do expect is that users will not reply to challenges that 
>come from strangers - if spammers start using someone's email 
>address for the "FROM"

I fear you are being optimistic.  Having spent much more time than I 
would like doing email customer support, I tend to think that you'll 
get two major categories of responses.
	1. Scared by challenge, doesn't reply to any.
	2. Finally figured out that challenges are okay, replies to all.
I can't offer concrete evidence for that.  But the number of viruses 
that spread via people opening attachments, and the number of people 
who ignore bounces completely, make me think that people just don't 
have a good understanding for what part of what they are looking at 
is important, and what part should be ignored.  When it comes to 
computers people really don't understand what's cause and what's 
effect.  It's the new magic.
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: Kee Hinckley <[email-masked]>
Date: Sun, 18 May 2003 22:14:34 -0400

>your logs.  Then additional data can be collected at a variety of

I will provide them.  However they are specific to CommuniGate Pro, 
and requiring turning on low-level SMTP monitoring so I can track the 
transactions.  Among other things that means that I'm getting about 
40MB of log files every night for those 40,000 bounces.  But I will 
provide the Perl libraries that I used to do the testing and generate 
the database.  Just give me a bit of time this week to do all the 
work that I was supposed to have been doing instead of this :-).

>evidence, and Kee's data is insufficient without wider deployment.

Yes.  It's limited data, and it's from an odd system.  Somewhere's 
email is somewhat abnormal compared to most companies or 
ISPs--although it probably looks more like a small ISP.

>meaningful and acurate data.

I don't think we're that far off.  The main issue is that spammer 
drop boxes get shut down--so the longer it is before you run the 
test, the less likely you are to get a valid email address.  (I've 
considered testing that assertion.  I might try periodically 
retesting addresses and seeing if disappear.)

At 2:49 PM -0400 5/18/03, Yakov Shafranovich wrote:
>Therefore, it is not possible to determine with certainty whether 
>these accounts actually existed. A better testing strategy would 
>actually send email to these accounts with the DATA command and 
>watch for bounce messages. However, spammers can always choose to 
>use a real email address as the return address and sending email to 
>valid accounts in itself may be considered spam by the recipients.

Yes.  I should have noted that this will report some number of items 
as valid when they are not.  However unless (as someone as asserted) 
the major ISPs are doing this, it doesn't impact the numbers we see 
for them.

Sending real messages is a bit risky for reasons other than just 
getting spam.  First of all, you can't send a bounce messages with no 
MAIL FROM (which you normally would) because you want to see the 
result.  Secondly, you run the risk of getting blacklisted as a 
spammer if you send too many test messages.  (Yes, it has happened to 
me.)
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: Vernon Schryver <[email-masked]>
Date: Sun, 18 May 2003 21:48:59 -0600 (MDT)

> spammer if you send too many test messages.  (Yes, it has happened to 

I don't want to imply or guess anything about those "too may test
messages," but it's worth noting something that too many people
conveniently forget or feel doesn't apply to their Special Needs.
Spam is never justified by any goal or desire of the spammer,
including any sort of surveying, testing, or avoiding spam.

---

From: Dave Crocker <[email-masked]>
Date: Sun, 18 May 2003 21:35:14 -0700

>> Can you show instances in which they say yes to messages they cannot deliver?

VS> I was skeptical of the report that Yahoo defers no-such-user responses
VS> until the DATA command, and so I made the obvious `telnet ... 25` test.
VS> I tried a perfectly valid message to an address that was very unlikely
VS> to exist.  The Rcpt_To command was answered with at 250 response.  It
VS> was not until the end of the DATA command that I got a 5yz.

Folks, the email operations world has increasingly moved away from
real-time processing of SMTP data.

Processing causes delays and SMTP client daemons are impatient. The
very, very wide range of network delays has compunded the effect of
these processing delays, constantly causing problematic timeouts.

So the tendency, these days, is to just say 'yes' in the protocol, take
the addresses and content, store them in a queue, and do any real work
after the connection closes.

d/
--
 Dave Crocker <mailto:[email-masked]>
 Brandenburg InternetWorking <http://www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>, <fax:+1.866.358.5301>

---

From: "Jon Kyme" <[email-masked]>
Date: Mon, 19 May 2003 08:56:36 +0100

> Can you show instances in which they say yes to messages they cannot

Yes. I'm assuming there is no valid yahoo local part corresponding to the
message ID of the mail you sent.

220 YSmtp mta154.mail.scd.yahoo.com ESMTP service ready
helo there
250 mta154.mail.scd.yahoo.com
mail from: <[email-masked]>
250 sender <[email-masked]> ok
rcpt to: <[email-masked]>
250 recipient <[email-masked]> ok

If you wanted to, it would be easy to try this with a set of random
"local-parts".  I normally see yahoo saying "No" after DATA.

---

From: "Jon Kyme" <[email-masked]>
Date: Mon, 19 May 2003 09:19:33 +0100

>after the connection closes.

The tendency may have been to move away from "real-time processing of SMTP
data", however with the increasing spam burden I wouldn't be surprised to
see the pendulum swing back. Only by making checks at the border can
traffic be rejected as near source as possible.
It's interesting that some MTAs nowadays offer hooks for highly
configurable checking during SMTP transaction (I'm thinking of the
local_scan interface in exim 4). The economics of handling spam
(and spam bounces) can make this very attractive even for high traffic
operations.

It seems sensible that a system should not accept messages that won't be
deliverable. The whole "accept anything / bounce it later" model is not
IMHO the wave of the future, rather expending CPU at the boundary MTA (and
providing the infrastructure to make checks) conserves resources later on.

Of course there are issues with implementation - I seem to remember hotmail
having problems with distributing their user database to their outward
facing MTAs in the past. However, we're all familiar with the sight of
systems failing under the demands of handling *bounces* (to non-existent
senders?) following massive spam runs (dictionary attacks) against them.

---

From: Kee Hinckley <[email-masked]>
Date: Mon, 19 May 2003 09:50:55 -0400

At 9:55 PM -0400 5/18/03, Yakov Shafranovich wrote:
>Well they can say yes even though it cannot be delivered - see RFC 
>2821, section 3.3.

Right.  But the question is--do they?

Fortunately it was easy to test.  And the answer is--yes.  Yahoo 
accepts email to addresses that do not exist.  I just made up several 
and got a 250 response to them all.  Needless to say, that throws the 
whole stats thing into complete disarray.

It appears that yahoo is only rejecting messages to addresses that it 
has already identified as being tagged by spammers.  (Thus the 553 
rather than 550 response.)  A check at a later date would be 
interesting, since they might have identified more.  But the response 
comes not just for abuse, but also for forgery, so that doesn't prove 
anything.  About the best I can do is mark hosts as to whether or not 
they normally do immediate rejects, and disregard those hosts that 
don't.
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: Kee Hinckley <[email-masked]>
Date: Mon, 19 May 2003 09:58:35 -0400

>including any sort of surveying, testing, or avoiding spam.

I was replying to mail sent to the [email-masked] hoax.  The 
message made it extremely clear (I spent a lot of time fine tuning 
it) in the first line that I was replying to mail they had sent me. 
It didnt' matter.

Initially I was doing a reply-all.  That got me in trouble real fast. 
I switched to just doing a normal reply--but I was still getting 
complaints and getting shut out by some servers.  So I gave up and 
bounced it with an extended bounce message.  That does not appear to 
have resulted in any spam complaints--although it gets ignored far 
more.

So.  Is it spam to send people email asking them to stop abusing your 
email server?  Not by any stretch I can imagine.  But in today's 
hair-trigger environment--it gets treated like it.
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: Vernon Schryver <[email-masked]>
Date: Mon, 19 May 2003 08:46:55 -0600 (MDT)

> email server?  Not by any stretch I can imagine. 

Nor in mine, provided you only ask one or two people instead of using
a so called "bitch list" of dozens to literally hundreds of addresses
of people at the other outfit who have nothing to do with it.

>                                                   But in today's 

Especially when the message is unexpected and comes from somewhere.com.

I wonder if you would have had the same results if you'd sent the
complaints from some other domain name.

---

From: mathew <[email-masked]>
Date: Thu, 22 May 2003 19:26:36 -0400

On Sunday, May 18, 2003, at 22:23 US/Eastern, Kee Hinckley wrote:
>> What I do expect is that users will not reply to challenges that come 
>> from strangers - if spammers start using someone's email address for 
>> the "FROM"
>
> I can't offer concrete evidence for that.

Consider the ActiveX warnings about unsigned code. How effective have 
they been at deterring the spread of web-installed spyware?

mathew

---

From: wayne <[email-masked]>
Date: Thu, 22 May 2003 22:40:46 -0500

> addresses to detect spam.  Their false positive and negative ratios

I run exim with "sender_verify_hosts_callback" enabled.  This causes
exim to connect back to the sending domain and uses a "MAIL FROM:<>",
"RCPT TO:<[email-masked]>" pair off commands to make
sure a potential bounce would be accepted.

It rejects quite a bit of email, but probably 60-80% of the spam
passes the test.

From what I can tell, the false positives are very low.  While systems
may not reject an invalid email address immediately, they shouldn't
ever reject valid email addresses.  The biggest problem I've had has
been with some yahoo groups mailing lists where Yahoo's "bounce MTA"s
were refusing connections.  In this situation, exim would give a
temporary failure, and Yahoo would try again, but only a few times.  I
ended up adding a special case exception for them.

-wayne

---

From: Vernon Schryver <[email-masked]>
Date: Fri, 23 May 2003 12:12:27 -0600 (MDT)

> From: wayne <[email-masked]>

> It rejects quite a bit of email, but probably 60-80% of the spam

One reading of that is that 60-80% of spam is not "forged."

Of course, that 
  - assumes most sending system answer 5yz for invalid users.  We know
     Yahoo answers Rcpt_To with 250 for some invalid addresses.

  - ignores accounts that have been terminated but cannot honestly
     be said to be "forged" when used by previous spammer owners.

  - interprets the claims that most spam is "forged" as meaning the
     the forgery is of invented sender addresses.

Tha last is reasonable, because if much mail were forged with valid
addresses, then you would expect to receive more than the rare spam
bounce....yes, of course some spam certainly is forged and some
addresses such as @somewhere.com get more than rare spam bounces.

I'm belaboring the obvious because I'd like to get the myth that most
spam is forged squashed before the spam problem is ended.

---

From: Yakov Shafranovich <[email-masked]>
Date: Fri, 23 May 2003 15:12:20 -0400

>spam is forged squashed before the spam problem is ended.

Some of spam is definitely being forged - the actual percentage varies 
depending on whom you ask (although someone like BrightMail might provide 
some real-world statistics). We should be looking at ways on dealing with 
it anyway regardless whether its the majority of spam. So whether it is a 
majority or not is irrelevant - but stopping forging will not solve the 
spam problem.

Yakov

---

From: Markus Stumpf <[email-masked]>
Date: Tue, 27 May 2003 02:22:27 +0200

> they have to, spammers could respond to challenges.

There would be an easy test method.
Modify the SMTP/Mail Server to send bounces not with an empty envelope
sender, but with a collector address. Also collect double bounces.
Then start counting. That way you will catch the bounces as they will
not be discarded by "store and forward" systems that have to accept what
will result in a double bounce.
By also recording the bounces you sent out and matching sender/dest
pairs the task would be rather trivial for people with e.g. spam traps.

I can't get my hands on a mailserver thats not in production and has
a large enough amount of mails for significant statistics.

As for the numbers:
From my experience /lot/ of sender addresses are faked. There is the
- keep username strategy
  abcdef@domain1 -> abcdef@domain2
  		 -> abcdef@domain3
  		 -> abcdef@domain4
		 [ ... ]
- [email-masked] strategy (I see this mostly with yahoo addresses)
    A customer got abused and they relayed 57653 spam mails.
    They mostly grouped it in 8 messages per sender account (about 8700 unique)
    They were formed:
	[email-masked]
	[email-masked]
	[email-masked]
	[email-masked]
	[email-masked]
	[email-masked]
	[email-masked]
    all 8700. And from all the bounces I had seen NONE of them was
    valid.
    I put the list up at
	http://www.lamer.de/download/spamsender.txt.gz
    in case anyone is interested. The format is  "nnn  address" where
    "nnn" is the number of times the address was used as sender address.
    (uniq -c)
- the - what I call them - "nolist" group.
  They use
      [email-masked]
      [email-masked]
      ...
  The bounces go back and are accepted (ad least they were before I put
  filters in) but they are abviously not evaluated as I always see
  the same (not existing/blocked) recipient lists tried over and over
  for months. So I am rather sure there is a bit bucket SMTP daemon
  accepting the messages.
- about 10% of the spam we (ought to) receive has
     - no valid A or MX record for the sender domain
       (slighty wrong figures because of the [email-masked] Sobig-A worm)
     - about 3-5% of the spam that bounces hangs in the queue (at least
       for a short time), because
	 - A/MX -> 127.0.0.1 or 0.0.0.0
	 - A/MX don't accept emails for 7 days period like
	      [email-masked]
	      [email-masked]
	      [email-masked]
	      [email-masked]
	      [email-masked]
	  ...

I doubt Vernons theory from my daily experience, but no, I have no real
numbers (s.a.)

	\Maex

-- 
SpaceNet AG            | Joseph-Dollinger-Bogen 14 | Fon: +49 (89) 32356-0
Research & Development |       D-80807 Muenchen    | Fax: +49 (89) 32356-299
"The security, stability and reliability of a computer system is reciprocally
 proportional to the amount of vacuity between the ears of the admin"
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg