Subject: Re: Last Call: Anti-Spam Requirements on an SMTP MTA to BCP

From: Owain Vaughan <[email-masked]>
Date: Mon, 2 Mar 1998 14:09:45 +0000 (GMT)

> I have a second more specific problem with the draft. Error 551 is
> mentioned several times..  eg
> 
> >       C  MAIL From: <[email-masked]>
> >       S  250 <[email-masked]>... Sender ok
> >       C  RCTP To: <[email-masked]>
> >       S  250 <[email-masked]>... Recipient ok
> >       C  RCTP To: <[email-masked]>
> >       S  451 <[email-masked]>... Denied due to spam list
> >       C  RCTP To: <[email-masked]>
> >       S  551 <[email-masked]>... Denied due to spam list
> 
> As I understand the current state of deliberations, 551 is being
> explicitly deprecated by DRUMS. I expect this should be 550 throughout.

Plus it says RCTP To: rather than RCPT To:

Owain

---

From: "Donald E. Eastlake 3rd" <[email-masked]>
Date: Mon, 2 Mar 1998 10:00:45 -0500 (EST)

Richard,

On Mon, 2 Mar 1998, Richard Clayton wrote:

> Date: Mon, 2 Mar 1998 12:48:14 +0000
> From: Richard Clayton <[email-masked]>
> 
> In message <[email-masked]>, The IESG <iesg-
> [email-masked]> writes
> >
> >The IESG has received a request to consider Anti-Spam Requirements on
> >an SMTP MTA <draft-lindberg-anti-spam-mta-02.txt> as a BCP.  This has
> >been reviewed in the IETF but is not the product of an IETF Working
> >Group.
> 
> ...
> 
> The most worrying example [in the draft] is this one:
> 
> >2.8. Verify "MAIL From"
> >
> >   The MTA SHOULD be able to perform a simple "sanity check" of the
> >   "MAIL From" domain and refuse to receive mail if that domain is
> >   nonexistent. To overcome temporary errors/problems in the DNS, 4xx
> >   Return Codes are strongly recommended; however the MTA MAY allow for
> >   Return Codes that show real DNS state - 4xx for temporary problems
> >   and 5xx for NonExistent domain.
> >
> >   In all honesty, please note that this requirement and ability is a
> >   mixed blessing and should be used with extreme care.
> >
> >   For early versions of spam spam software it does provide quite some
> >   relief, since that software generates mail with completely bogus
> >   "MAIL From" that will never even get into the system.
> >
> >   On the other hand, sites with weak DNS connectivity may find their
> >   legitimate mail having problems reaching destinations due to DNS
> >   timeouts. However, since DNS information is handled asynchronously
> >   and is cached even though the initial requester has given up, chances
> >   are high that the necessary information is there at a later attempt.
> >
> >   ...
> 
> I do note the caveat of "extreme care", but the author seems to be more
> concerned with poorly connected DNS, rather than a DNS configuration
> which is just plain wrong!
> 
> In RISKS (http://catless.ncl.ac.uk/Risks/19.24.html) Garfinkel discusses
> what happened when a small New England ISP (vineyard.net) tried this
> approach. Bottom line was that they stopped because they were bouncing
> too much genuine material, and those bounces were from sites with
> perfectly accessible, but incorrect DNS entries.

I believe you are incorrect.  The test being performed by Vineyard.NET at the
referrenced URL was the test of determing that the IP address from which the
SMTP connection came had an inverse entry (in .in-addr.arpa for IPv4).  This
has nothing to do with the "MAIL From" address, which is arbitrary. 

Furthermore, the draft merely proposes as BCP the availability of the
capability of testing that the MAIL From address has a domain name which
exists in the DNS, it does not specify as BCP the activation of this testing
capability. 

With the tendency of spammer vermin to multiply, people running SMTP servers
on the Internet need whatever capability they can get to tightening
constrains on the mail they receive and at least narrow the range of
behaviour of the abusers so they can be more easily tracked and appropriate
counter measures taken. 

There are exceptions, such as SMTP servers run by NICs that may be receiving
mail related to DNS configuration problems, where you would not want to
impose such checks.  But I believe that in fact, not only do you usually want
to check the MAIL From address so that it is at least possible that you could
reply to it (unless it is the specially permitted null address for errors),
you should also optionally be able to impose the check that Vineyard.NET
tried, of seeing if there is an inverse DNS entry for the SMTP source IP
address, and if you desire of going a step further and testing that the
domain name gotten from the inverse look-up forward translated to a set of
addresses including the one from which you are getting the SMTP connection. 

> I am concerned that elevating this type of filtering from a local policy
> which meets local needs to "BCP" status, is likely to have a long term
> and damaging effect on a fundamental protocol.
> 
> Refusing to accept mail because the senders DNS is wrong or inconsistent
> is completely at odds with the Fundamental Principle of being liberal in
> what one accepts.

That is not some immutable Fundamental Principle overriding all others, it is
a general principle which needs to be adjusted for individual circumstances. 

This action proposed in the draft results in clear indications back to the
sender that their mail did not get through so they can try different mail or
other forms of communicaitons.  The "damage" to the mail protocol is being
done by the thieves who send bulk unsolicited email.  Get them to stop and I
think you will find that not only the extemely mild measures in this draft
but also the much more draconian but necessary actions such as the Real Time
Block Hole list (which, last I knew, EUNet among many other subscribed to and
enforces) would fade away. 

> My apologies if I should have been discussing this earlier or elsewhere.
> 
> -- 
> richard                      richard.clayton    @    T U R N P I K E .com
>                                                      tel: +44 1306 732300
> "Assembly of Japanese bicycle require great peace of mind" quoted in ZAMM

Donald
=====================================================================
Donald E. Eastlake 3rd     +1 978-287-4877(tel)     [email-masked]
   318 Acton Street        +1 978-371-7148(fax)     [email-masked]
Carlisle, MA 01741 USA     +1 703-620-4200(main office, Reston, VA)
http://www.cybercash.com           http://www.privacy.org/ipc

---

From: Ned Freed <[email-masked]>
Date: Mon, 02 Mar 1998 14:55:41 -0800 (PST)

> I have no love of unsolicited junk mail, and there is much good advice in
> this document.

> However, to endorse all of its suggestions as "best current practice"
> without a detailed analysis of the downside of implementing them does not
> seem to me to be wise.

I'm afraid I must agree with this assessment. (And I have said as much in
private mail to the IESG in which I objected to advancement of this document to
BCP in its present form.)

> exists in the DNS, it does not specify as BCP the activation of this testing

This is quite correct but also beside the point. The IETF's criteria for
evaluating protocols revolve around implementation and use.  In particular,
interoperability issues that arise from implemention and use are of paramount
importance.

As such, it is not sufficient for a specification to be correct in a narrow
sense; it must also avoid language that readily admits incorrect
interpretation, implementation, and use. And the unfortunate fact of the matter
is that a document describing facilities of the sort described here will be
taken by managers as justification for using these facilities in highly
inappropriate ways. This will then lead to all manner of interoperability
problems. We currently are besieged by interoperability problems brought about
by misguided attempts to block spam; I currently spend more of my time dealing
with the unintended consequences of spam blocking than I do dealing with spam
itself.

None of this is unfixable; the addition of text describing the problems
with these mechanisms and what must not be done with them will eliminate
my objections to the document.

> The most worrying example is this one:

> perfectly accessible, but incorrect DNS entries.

While I agree with the assessment that this document needs careful review and
the addition of a fair amount of text describing the possible consequences of
inappropriate use of the various mechanisms it describes, I do not agree that
his is the biggest problem. As Donald Eastlake points out:

> The test being performed by Vineyard.NET at the

There is in fact a big problem with validating MAIL FROM in this way: In their
zeal to implement the best possible blocking, many managers fail to take into
account that it is perfectly legitimate for the MAIL FROM field to be empty.
Indeed, current Internet standards REQUIRE that MAIL FROM be left blank in
[non]delivery notifications; failure to do so can result in nasty message
loops. Yet the requirement that logically follows (MTAs MUST NOT block messages
with blank MAIL FROM fields) from this is mentioned nowhere in this document.

The leeway that is apparently given to treat transient DNS problems as
permanent, fatal errors in this part of the specification is also unacceptable.

> behaviour of the abusers so they can be more easily tracked and appropriate

This is also quite correct, but we also have a responsibility to fully and
completely document the consequences of using these tools. 

Another common error in setting up these tools is the inadvertent promotion of
something done to block a transient event to permanent site policy.
Specifically, a spam incident happens, rules are installed, the incident ends,
but the rules remain in place. Eventually nobody remembers what the rules are
there for and thus doesn't feel comfortable with removing them.

For this reason requirements should also be placed on MTAs by this document to
either provide a means of expiring old rules, implement "holes" in general
rules (e.g., postmaster to postmaster mail works but everything else is
blocked), or both.

Vernon Schryver also writes:

> That battle is over.  Many large outfits are quite picky about the
> mail_from domain.  If you don't ensure that your system sends with good
> mail_from values, you cannot talk to a lot of the Internet.  Whether
> that is good or bad and whether or not it is in a BCP is irrelevant.

There is a big difference between what sites are currently doing (which can
change very quickly) and what the IETF blesses as recommended practice (which
has a way of hanging around for a long time). And again, like it or not, the
distinction between recommending that MTAs provide facilities and people
actually using those facilities is going to be lost almost immediately.) The
problem doesn't like in what is recommended, it lies in the lack of discussion
of what not to do.

				Ned

---

From: "Dick St.Peters" <[email-masked]>
Date: Mon, 2 Mar 1998 22:12:39 -0500

> > perfectly accessible, but incorrect DNS entries.

We're _almost_ a small New England ISP - Vermont is only 20 miles east
of here - and we require that domains resolve.  We have for some time.

So far today, this has blocked 57 emails "from":

   1 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>
   9 <@gateway.24hrplaymates.com.244.216.208.in-addr.arpa:[email-masked]>
  13 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>
   3 <[email-masked]>
   2 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>
   1 <ml-pgh-hci-request@list-processor>
   1 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>
   2 <[email-masked]>
   1 <[email-masked]>
   1 <[email-masked]>

It's been worth every cycle, IMNSHO.

For our clients' domains we have dual off-site secondaries, one her in
NY state and one in Colorado, so if people have their mail bounce
because their domains temporarily don't resolve, I have a solution for
them :)

Of course, requiring 'from' domains to resolve might force everyone to
have redundant offsite secondaries, and we'd lose that competitive
edge.  Perhaps that's not a Bad Thing ...

--
Dick St.Peters, [email-masked] 
Gatekeeper, NetHeaven, Saratoga Springs, NY, 1-800-910-6671 (voice)
Saratoga/Albany/Amsterdam/BlueMountain/Cobleskill/Greenwich/
GlensFalls/LakePlacid/NorthCreek/Plattsburgh/...
	  First Internet service based in the 518 area code

---

From: Ned Freed <[email-masked]>
Date: Mon, 02 Mar 1998 21:12:15 -0800 (PST)

> of here - and we require that domains resolve.  We have for some time.

Um, what is your point, exactly? I'm well aware that this approach can be
successful at blocking some sorts of spam. I never said that it wasn't; indeed,
I disagreed with Richard Clayton's assertion that requiring this facility is a
problem with the present document.

My problem with the document's description of MAIL FROM validation is that no
mention is made of the need to make an exception when the MAIL FROM field is
completely empty. And this same problem of lack of guidance in how to use these
facilities is a general problem with the document as it standards -- you will
not find a single MUST NOT or even a SHOULD NOT anywhere in it.

Now, if it was your intent to refute Richard Clayton's assertion that's fine,
but then why was your reply directed at me?

				Ned

---

From: "Dick St.Peters" <[email-masked]>
Date: Tue, 3 Mar 1998 12:44:34 -0500

> but then why was your reply directed at me?

Sorry ... I no longer had a copy of Clayton's message; things got a
bit frantic here for awhile; a telco installed $100/day in digital
phone service at a remote site without telling me.  I called asking
for an estimated install date ... "oh, we installed those lines last
week" ... apparently on the very day I had last called asking for a
date estimate.