Subject: Re: Last Call: Anti-Spam Requirements on an SMTP MTA to BCP

From: [email-masked] (Vernon Schryver)
Date: Mon, 2 Mar 1998 10:43:44 -0700

> From: Richard Clayton <[email-masked]>

> ...
> >an SMTP MTA <draft-lindberg-anti-spam-mta-02.txt> as a BCP.  This has

> ...
> The most worrying example is this one:
> 
> >2.8. Verify "MAIL From"
> >
> >   The MTA SHOULD be able to perform a simple "sanity check" of the
> >   "MAIL From" domain and refuse to receive mail if that domain is
> >   nonexistent. To overcome temporary errors/problems in the DNS, 4xx
> ...

That battle is over.  Many large outfits are quite picky about the
mail_from domain.  If you don't ensure that your system sends with good
mail_from values, you cannot talk to a lot of the Internet.  Whether
that is good or bad and whether or not it is in a BCP is irrelevant.

Note that if your mail_from value is bad, bounces are likely to not
work.  Anyone who actually tries to keep up with postmaster mail for
other than a tiny domain hates people who send undeliverable mail with
invalid return addresses.  People running mailing lists which generate
mail_from values that do not work (and not just bad hostname but also
bad usernames) get my goat.

> I do note the caveat of "extreme care", but the author seems to be more
> concerned with poorly connected DNS, rather than a DNS configuration
> which is just plain wrong!
>
> In RISKS (http://catless.ncl.ac.uk/Risks/19.24.html) Garfinkel discusses
> what happened when a small New England ISP (vineyard.net) tried this
> approach. Bottom line was that they stopped because they were bouncing
> too much genuine material, and those bounces were from sites with
> perfectly accessible, but incorrect DNS entries.

I've found it useful to reject mail with mail_from values that cannot
themselves be resolved to A or MX records (not just an NS record) or
parent domains (of course not TLD's) that cannot be resolved to A or MX
records.  That accepts mail from new hosts in valid domains.  This rule
does reject a little legitimate email, but it rejects a much larger
amount of spam.  Sgi.sgi.com moves about 200K messages/day.  That
machine rejects 3,000-14,000 spam/day, and uses the DNS rule for 10-20%
of those rejections.  The false positive rate for the DNS rule is less
than 1%.  (Automated body filters are the most effective scheme on the
machine.)  (The spam rate I'm seeing has been dropping irregularly
since Christmas.)

] From: "Donald E. Eastlake 3rd" <[email-masked]>

] ... 
] I believe you are incorrect.  The test being performed by Vineyard.NET at the
] referrenced URL was the test of determing that the IP address from which the
] SMTP connection came had an inverse entry (in .in-addr.arpa for IPv4).  This
] has nothing to do with the "MAIL From" address, which is arbitrary. 

Many people think that completely bogus check is reasonable.  Other
popular nonsense involves requiring the HELO value to equal the reverse
DNS value of the peer's IP address or requiring the mail_from value to
equally the HELO string.  It wouldn't hurt to document the silliness of
those notions.  I would have thought they're so obviously wrong that no
one would need to be told, but they are popular.

Vernon Schryver,  [email-masked]

---

From: Gunnar Lindberg <[email-masked]>
Date: Tue, 3 Mar 1998 17:48:03 +0100 (MET)

>Date: Mon, 2 Mar 1998 12:48:14 +0000

The first part of the mail has already been discussed and I assume
there is more to come. It would be nice to get <[email-masked]>
on the Cc: list (I forgot in my prevoius reply).

>Refusing to accept mail because the senders DNS is wrong or
>inconsistent
>is completely at odds with the Fundamental Principle of being
>liberal in
>what one accepts.

You may actually view if differently: If you accept mail whose
"MAIL From:" does not resolve in the DNS, you actually take on
a responsibility to manually trace the sender in case "RCPT To:"
is non-existent or fails by any other reason. This is especially
important if you've accepted to be a Relay (secondary MX etc).

If the response is "4xx" you actually leave that problem to the
sending host - you don't even touch mail that you cannot Error
Return. In the long run this kind of "pushes back" nearer to the
(incorrect) sources.

Liberal? Nope. Strict? Yes. Anti-spam? Probably. Careful?
Yes, I honestly think so. Correct? Well, *I* think so...

>As I understand the current state of deliberations, 551 is being
>explicitly deprecated by DRUMS. I expect this should be 550
>throughout.

I assume this is from draft-ietf-drums-smtpupd-06.txt. Now using
a value from one draft in another draft is risky, but "550" is
"equally much 5xx" for this, so I'll be happy to change.

>From: Owain Vaughan <[email-masked]>

>Plus it says RCTP To: rather than RCPT To:

Thanks for catching that typo!

	Gunnar

---

From: Gunnar Lindberg <[email-masked]>
Date: Tue, 3 Mar 1998 19:23:10 +0100 (MET)

Ned, thanks, you really scared me...

I'm too new to the IETF process to know, but the document has been out
for some time (and was even briefly presented at the BOF in DC) and
yet not very many people have had/taken time to read and comment. It
could be either because it's all set and done or that the people one
really want to review it haven't had that time (yet). Obviously the
IESG had similar thinking and issued the Last Call and obviously it
was the latter case.

I do agree that a document that can be erronously interpreted also
will be interpreted that way, and thus we need to make sure that the
ambiguities that eventually remain are minimal.

So, I assume the IESG will decide accordingly and then we (I?, who?)
will have to organize that review and come back with a more complete
and better prepared document.

	Gunnar

>Date: Tue, 03 Mar 1998 09:13:33 -0800 (PST)
>From: Ned Freed <[email-masked]>
>Subject: Re: Last Call: Anti-Spam Requirements on an SMTP MTA to BCP
>To: Gunnar Lindberg <[email-masked]>
>Cc: [email-masked], [email-masked]
>Message-id: <[email-masked]>

>> First of all I assume I will get a copy from the IESG with your
>> comments so that they can be included. I saw no "Cc:" (or was the
>> suggestion to move the document to /dev/null and go home?).

>I do not want to see this document abandoned. I think it is an excellent
>starting point for something good. My only objection is over the document
>moving forward in its present form.

>My suggestion was that the document not be accepted until it has received
>comprehensive and thorough review by a significant number of experts on email
>in general and SMTP in particular. I turned up several problems that I believe
>need to be addressed besides the MAIL FROM:<> issue; I have no reason to
>believe that additional review done by persons other than myself won't turn up
>other problems as well.

>I don't think a WG is needed to do this, but that's just my take on it; it is
>the IESG that will decide. Nevertheless, I believe additional review is
>required before it is appropriate for this to become a BCP.

>> It's quite true that "MAIL From: <>" is not explicitly mentioned in
>> the document and I agree that it should (and maybe not only in the
>> context of sanity checks of "MAIL From:" - some implementations
>> actually seem to do Relay Authorization based on "MAIL From:", which
>> I personally think is worse and which can give "interesting results"
>> when faced with "MAIL From: <>").
>    
>Indeed, relay checks based on MAIL FROM is yet another problem area that needs
>to be mentioned.

>> Weak DNS connectivity can be a problem, but is less of a problem than
>> one may think. What seems to happen is that the Name Server pointed
>> out by my resolver sends out the query and even if my MTA gives up
>> and exits before it gets an answer, the information is cached when
>> it eventually arrives, so that it's there at the next attempt. OK,
>> there are still DNS domains with 5 min cache life time *and* weak
>> connectivity... Can't always win. Sigh.

>Whether or not this is a problem isn't relevant to my objection; the point I'm
>trying to make is that the discussion of various issues is inadequate. I've
>said it before and I'll say it again: I completely agree that tardy DNS
>behavior is rarely a problem in practice with these sorts of checks when the
>checks are properly implemented. Pathological cases can develop where the retry
>rate for the message interacts with DNS cache time outs in exactly such a way
>as to guarantee that a message is never delivered, but this is far more likely
>to be associated with client DNS queries rather than server DNS queries.

>> The suggestion in the draft is to return 4xx at most occasions. Not
>> everybody agrees to this, but when it comes to DNS based checks I
>> still argue that's a good thing. Of course DNS TempFail MUST result
>> in a 4xx response code, but doing so even for an Authoritative NamE
>> actually gets away with many DNS sync problems etc.

>Yes, but nowhere in your document do you actually say this. What you do say is:

>   Return Codes are strongly recommended; however the MTA MAY allow for
>   Return Codes that show real DNS state - 4xx for temporary problems
>   and 5xx for NonExistent domain.

>And elsewhere you say:

>   However, 4xx Temporary Errors may have unexpected interaction with
>   MX-records. If the receiving domain has several MX records and the
>   lowest preference MX-host refuses to receive mail from a certain
>   "MAIL From" domain with a "451" Response Code, the sending host may
>   choose to - and often will - use the next host on the MX list.

>And:

>   10) MUST be able to configure for different Return Codes for
>       different rules (e.g. 451 Temp Fail vs 551 Fatal Error).

>Now, being the knowledgable person that I am, I know what you're trying to say
>is that temporary failures MUST return a 4xx and permanent failures SHOULD
>return a 4xx but MAY return a 5xx.

>But it is also possible to read this as saying that an MTA should return 4xx on
>temporary failures but MUST allow itself to be configured to return 5xx, and
>moreover cases arise where such configuration is the right thing to do. And I
>have way too many years of experience with standards work not to think that
>this is EXACTLY how some people will read it, and will implement it, and all
>sorts of operational problems will result.
> 
>> In fact, it may take a faster keyboard user then I am to keep the
>> document up to speed with spam evolution.

>This is actually the one usability issue I don't especially care if the
>document addresses. Sure, it is true that none of the measures this document
>describes stand much chance of being effective in the long term -- in fact, the
>documenting of them and widespread adoption of them will inevitably lead to
>them becoming less effective.

>But this is the way of all standards and most especially BCPs -- the problem
>they seek to solve either vanishes, mutates, or a better solution comes along.
>We don't think its necessary to put disclaimers as to the inevitability of
>obsolescence in other standards documents so I see no reason to require
>discussion of it here either. The BCP label is enough of an indicator that this
>is transient stuff likely to require updating on a regular basis.

>In conclusion, I do believe that the value of having such a document will more
>than offset the harm that will inevitably come from people misreading it and
>using the facilities it provides improperly. But this in no way diminishes our
>responsibility to attempt to minimize the chances of this happening by being
>thorough and complete.

>				Ned

---

From: [email-masked] (Vernon Schryver)
Date: Tue, 3 Mar 1998 12:01:33 -0700

I agree that the document is too immature to receive the BCP label.
Best Current Practice should not have connotations of "this is the best
we've thought of in the 10 minutes we've considered the issue."  Spam
filtering is too new for there to exist a concensus on what to do about
it.  Consider the fairly recent hoohah in this mailing list about AOL's
rejection of source routes in mail_from values.  The ideas as well as
the words in a BCP should have a chance to mature before being
blessed.  As has been repeatedly said, the document is not sufficiently
clear, precise, or complete.

> From: Gunnar Lindberg <[email-masked]>

>     o   But, spam evolves too:

> ...
>         This is from/about the "Rapid Fire Mail Server (RFMS)", which
>         people seem to sell for $200. Second/Third Generation spam.

I view the "Rapid Fire Mail Server (RFMS)" as throw-back.  It has not
required the slightest changes in the SGI filters.  It is doing the
same as Mustashi's garbage did months ago (sending from <>) except it
is too stupid to relay, and except that the RFMS output does not have
the twiddles that may have been intended to foil body matching.  The
empty From: and To: lines are silliness that make it easier to filter
with procmail, and that anyone with half a brain can see how to fix.
The only near-novelty of RFMS that I see is that it is being sold more
aggresively than other spamware as part of a MLM/pyramid scheme, and
that its sales blather mentions laws against forgery.

] From: "Dick St.Peters" <[email-masked]>

] ...
] OTOH, the [email-masked] address looks like a case of legitimate
] but misconfigured mail - the system trying to send us the mail is
] indeed an epa.gov mail relay.  Perhaps our rejecting the mail will
] result in someone helping 'amy' correct her mail configuration, and
] heer correspondents will be able to reply to her mail.

The rules I use would pass that address, because r01.epa.gov is
defined.  As I said before, I think it's useful to pass hostnames that
seem to be new systems in legitimate domains.

---

From: "John S. Quarterman" <[email-masked]>
Date: Wed, 04 Mar 1998 08:44:07 -0600

>At 02:16 PM 04-03-98 +0100, Sascha Ignjatovic wrote:
>>IT MUSST BE ABLLE FOR BIG COMPANYS TO SUE SUCH SPAM COMPANYS WHO USES
>>THEIR DOMAIN NAME-NOT ACUOUNTS  AND TRADEMARKS TO 
>
>There was a recent case in Texas (I believe) where a small-time spammer
>caused significant trouble for an ISP and a small company by using that
>company's domain name as a fake return address.
>
>The court found in favor of the small company that the spammer had damaged
>them and misappropriated their resources.
>
>The spammers can be slammed legally if the companies being abused will just
>do it.

This is all true.

For details about that case, see http://www.mids.org/nospam/.

Thanks,
John
-- 
John S. Quarterman <[email-masked]>
President, Matrix Information and Directory Services (MIDS)
[email-masked], http://www.mids.org, +1-512-451-7602, fax: +1-512-452-0127
1106 Clayton Lane, Suite 500W, Austin, TX 78723

---

From: Gunnar Lindberg <[email-masked]>
Date: Thu, 5 Mar 1998 17:23:16 +0100 (MET)

Sascha, please. This MAY be a legal problem, but it involves legal
systems in MANY countries and for the time being that is unclear, to
say the least. Until that is resolved we have to use some technical
methods, to hold back the best we can. The draft that started this
thread contains one particular take on that.

If you have been abused by someone from somewhere, please contact
them directly and get that problem resolved.

But, please, p l e a s e, refrain from spamming the IETF list. We
have enough problems with the spammers themselves and we don't need
to do that to ourselves.

Thank you,

	Gunnar

>Date: Thu, 5 Mar 1998 16:50:05 +0100 (MET)
>From: Sascha Ignjatovic <[email-masked]>
>To: William Allen Simpson <[email-masked]>
>cc: [email-masked]

>On Thu, 5 Mar 1998, William Allen Simpson wrote:

>> I sent Jonathan Clark some examples of worldnet relaying and forged
>> headers from att subscribers....
>> 
>> In particular, mtigwc04.worldnet.att.net ([204.127.131.33]) seems to be
>> wide open!
>> 
>> [email-masked]
>>     Key fingerprint =  17 40 5E 67 15 6F 31 26  DD 0D B9 9B 6A 15 2C 32

>but anyway  

>it is a legal problem hier and not only a technical 

>when some companys uses a name from some other company in a abusive way

>the abused company hase the RIGHT and DUTY to FIGHT this over

>if they dont and just trys to close "holls" it solves so not the real
>problem 

>the abusive missusing of companys names and trademarks 
>this problem remains unsolved with "closing the holls"

>they should also close the "juridical holl" by joining a cooperation and
>process with some other companys and organisations/states and develope 
>policy and jurisdiction wich would prohibite i the future missusing 
>and fooling their good names and property

>thats the wholle point hier 

>after that is solved the technical one may be easyer to solve 

>you just have to filter out the fantasie domains who hase no dns entry and
>respons 

>it musst be ilegal to use other companys domain names to do something
>nasty/ilegal and so to cover your the origine of the act
>altough on all spam mails you would have adress and bank acount of the 
>abuser

>pelase exuce me i started the discussion from the point of comon sense and 
>i am no a technician  but i hope that in this way i contribute just a
>little to handle this broad and dificult topic of spaming 

>thanks
>sascha