Subject: Re: namedroppers, continued

From: "Doug" <[email-masked]>
Date: Tue, 7 Jan 2003 13:33:28 -0500

Hello Mr. Wood,

----- Original Message -----
From: "Lloyd Wood" <[email-masked]>
To: "Doug" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Tuesday, January 07, 2003 7:05 AM
Subject: Re: namedroppers, continued

> Doug has rediscovered the idea of closing open mail relays to
prevent
> unauthorised use by outsiders sending to outsiders. This was a big
> thing in the early 90s when email became popular.

This may seem to be a bit basic for some of the people who have worked
on this problem for years. My intention was not to prove that I had
the latest and greatest solution to the spam problem. It was to get
the ball rolling in an open discussion forum and present my ideas on
the topic in the hopes that someone who knew more than me on the topic
would as well.

>
> Doug has also come up with the idea of adding the IP address of the
> originating client machine (not necessarily using SMTP) in a header
> so that an attempt can be made to identify it - e.g. Hotmail has
done
> that for years.

After examining the headers of many of the spam advertisments I get
and trying to contact the administrator of the network it came from I
find that it is usually futile because the network doesn't exist and
the IP information is incorrect. I also find that most use false
sender and reply address information (in an attempt to keep recipiants
from filtering them). This makes it hard (at least for me) to do
anything about them. I have experimented with filters for subject
wording but this unfortunately hits on some of my wanted email as
well. This reduces my ability to to block them on the receiving end.
Even if I could it doesn't help the net congestion they cause or do
anything about the processing time it is using across the net. These
things leads me to propose that a more global solution needs to be
implemented. The problem here is that when you bring this up for
discussion in a professional environment like this one people don't
want to discuss it. Instead they consider it a problem that has no
solution and just want to forget about it.

>
> L.
>
> missing mail admin experience, I think.

Very true. I have never administered anything other than my http and
ftp servers. I have thought of turning on the mailserver but I do not
know enough about administering it yet and I really have no need for
it. I certainly hope that nobody thought I actually ran my own mail
server because I was not my intention to pretend that I did.

It is nice to see someone with more knowledge and/or experience on the
topic than me taking the time to think (and talk) about it.

Thanks for the input,
Doug
Asking questions, presenting possible solutions, and learning from
mistakes is how we get solutions.
---------------snipped previous for sake of size------------------

---

From: [email-masked]
Date: Tue, 07 Jan 2003 15:26:55 -0500

On Tue, 07 Jan 2003 13:33:28 EST, Doug said:

> anything about them.

The trick here is to remember that except for the relative few spammers that
are advocating a religious/political/philosophical viewpoint (a la "Uncertainty
Principle is Untenable!"), the spammers *WANT* you to be able to contact them
via *some* means - they can't extract money (their usual goal) from you if
you can't get back to them.

Moreover, it has to be relatively simple to find - it has to be simple enough
that even a victim who doesn't have enough kloo to stop to wonder why the
"confidential and private" Nigerian scam arrived via spammage can figure out
how to get aboard....

-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

---

From: [email-masked]
Date: Tue, 07 Jan 2003 15:26:55 -0500

---

From: "Tony Hain" <[email-masked]>
Date: Tue, 7 Jan 2003 14:39:02 -0800

Doug wrote:
> ...
> After examining the headers of many of the spam advertisments 
> I get and trying to contact the administrator of the network 
> it came from I find that it is usually futile because the 
> network doesn't exist and the IP information is incorrect. I 
> also find that most use false sender and reply address 
> information (in an attempt to keep recipiants from filtering 
> them). This makes it hard (at least for me) to do anything 
> about them. I have experimented with filters for subject 
> wording but this unfortunately hits on some of my wanted 
> email as well. This reduces my ability to to block them on 
> the receiving end. Even if I could it doesn't help the net 
> congestion they cause or do anything about the processing 
> time it is using across the net. These things leads me to 
> propose that a more global solution needs to be implemented. 
> The problem here is that when you bring this up for 
> discussion in a professional environment like this one people 
> don't want to discuss it. Instead they consider it a problem 
> that has no solution and just want to forget about it.

An approach that is more effective than scanning for content is to
simply block connections from the last hop in the SMTP chain before
yours. This kills both direct spammers as well as open relays. The list
can be long (mine is ~ 256 /24's for a private little mailer), but that
is a tradeoff against how much space you want to block at a time. On
several occasions I have considered putting in 61/8, 200/6, & 210/7,
because that would remove 3/4 of the list, but that also creates a
guilt-by-association for people with no control over their address space
or those who are abusing it. 

While this approach avoids having the content traverse the wire, some of
the machines are tenacious, as yesterday's log shows. 
http://www.tndh.net/~tony/ietf/2003-01-05-log.txt

Tony

---

From: "Tony Hain" <[email-masked]>
Date: Tue, 7 Jan 2003 14:39:02 -0800

Tony

---

From: "Daniel Pelstring" <[email-masked]>
Date: Tue, 7 Jan 2003 20:54:35 -0500

Doug,

    This topic comes up quite often.  It has been discussed at length many
times.  Do not take criticism of your ideas too harshly, it is just that
most people here have seen these same solutions proposed by many people over
many years.  It gets a bit old and, patience gets a bit short.  A very quick
summary of the solutions proposed during these discussions is as follows:

A) Anybody can send mail to anybody (what we currently have.)

B) Mail servers check that the sender is authorized to send mail to the
users of its system.

C) Sending mail requires some sort of computationally expensive to compute,
easy to check, data to be sent along with messages (probably based on sender
address, receiver address and message)

    Unfortunately, B requires that there be some way to authenticate users
of other systems.  The only way that has been thought of to do this is to
have some central authority to do this.  While it is certainly possible,
this is thought to be a bad idea for many reasons, the most prominent of
which is the potential for abuse.  Lacking a central authority, this can be
done on mail servers already, with no need to involve the IETF (an example
would be whilelists.).

    Unfortuantely, C would have to be computable in a reasonable amount of
time in order not to be a major headache.  This means that the spammers
large cluster of powerful computers would still be able to send many
messages.  It would also wreak havoc with mailing lists.  It would also
become easier to compute as time goes on, due to the ever progressing nature
of computers.  This does not even take into account that we would be
creating a standard with the sole purpose of inefficiency, which probably
makes most engineers wince and, although that may not seem like such a
horrible idea right now, this standard could survive far into the future,
with implications yet to be known (keep in mind the scale on which these
standards will be implemented.)  If another solution is implemented in the
future, this may come back to bite us.

    At the moment, it seems the IETF consensus (notice the *seem* here, this
is my personal opinion taken from reading the discourse on these lists) is
that the best solution is A.  At some point another solution may hold the
majority favor, but that time is not now.

    Of course, it is possible that there is a solution that has not been
thought of, in which case it is a bad idea to discourage people from
thinking about it.  Know, however, that this list contains many extremely
arrogant (many of them justifiably so) experts in their chosen field, who
dislike their time being wasted.  I encourage you, if you have further
ideas, to do the required research to see if it would work before sending a
message and, be very thorough.  Also know that if you choose to get into a
flame war with people on this list, that some them have likely been doing
this for decades and, are probably much better at it than you are.  It is
likely to do nothing but frustrate you.  If you have another idea and, have
questions about how to check that it is at least feasable, I would be happy
to point you in the right direction.

-Daniel Pelstring

---

From: "Daniel Pelstring" <[email-masked]>
Date: Tue, 7 Jan 2003 20:54:35 -0500

Doug,

-Daniel Pelstring

---

From: John C Klensin <[email-masked]>
Date: Tue, 07 Jan 2003 18:22:06 -0500

--On Tuesday, 07 January, 2003 13:33 -0500 Doug
<[email-masked]> wrote:

>> unauthorised use by outsiders sending to outsiders. This was
>> a big thing in the early 90s when email became popular.
> 
> This may seem to be a bit basic for some of the people who
> have worked on this problem for years. My intention was not to
> prove that I had the latest and greatest solution to the spam
> problem. It was to get the ball rolling in an open discussion
> forum and present my ideas on the topic in the hopes that
> someone who knew more than me on the topic would as well.
>...

Ok, Doug.  Let me take a shot at simultaneously explaining the
problem here and  why your suggested solutions are getting such
resistance.  Perhaps a different approach may succeed where
Lloyd's didn't (not his fault, I'm sure).

Almost all of the measures you have suggested have serious
side-effects or critical prerequisites.  In the last analysis,
most of us would rather put up with a little spam than pay the
prices involved.  Others are sufficiently fed up with spam that
they are willing to consider some very radical changes to how we
use email.  But, regardless of how that comes out, the decisions
have been fairly explicit: people have thought of your
suggestions, and others, and their impact, and have made fairly
explicit decisions about preferences.  My comment about X.400 of
a few weeks ago was intended to address those issues, but
apparently made a reference too far in the past, or too subtle,
for some of the people who have been participating in the
discussion.

The people who have been through it are reluctant to start the
discussion again because the dead horse has been kicked into an
unrecognizable pulp. There has been a good deal of discussion,
in many archives, that it would be good for you to read.  If you
then come up with a _new_ idea that doesn't revisit the old
problems, many of us would be enthused about listening.  It is
only the old and discredited proposals that are met with
intolerance.

Let me take a closely-related pair of examples from your note.
As I understand it, you would like ISPs (really email providers)
to take responsibility for authenticating their customers.  And
you would like IP addresses shown in mail header traces to be
reliable.  Ok.  Those two things turn out to be much more about
trust relationships than they are about protocols: one can make
major changes to protocols without changing the trust
relationships at all, and can accomplish those things without
any changes in the protocols.  But, assuming that you aren't
willing to trust anyone who can operate a mail-sending protocol,
there are only a few ways that you can trust the source and
origins of email.  

For example, we could, as a society, start licensing email
providers, require each licensed provider to accept mail only
from other licensed providers and from users they can
authenticate (with severe penalties for violating those rules).
That would make folks who would like to go back to business
models in which they charge for every item of mail very happy.
It would make many of the rest of us unhappy.  But it would
work... and, while some protocol enhancements might make it
easier to support, nothing is really required beyond SMTP as it
exists today.

Similarly, you could decide to work with an email mailbox
provider who would refuse to accept mail from any site with
which it didn't have an agreement about user authentication and
traffic and which, were relaying permitted, would insist that
any site from which it accepted a relayed message impose the
same requirements on its users.  With some prototype agreement
forms, this could be a way to build up a trust web similar to
the licensing one, using a web of bilateral agreements rather
than governmental action.  

But either of those approaches would result in less legitimate
mail getting through.  For instance, I run my own mail servers
and, as far as I know, can authenticate anyone who originates
mail here.  But I have neither the resources nor the inclination
to participate in a large collection of bilateral contractual
agreements, nor to start applying for licenses.

The issues with those bad IP addresses are similar.  Most are
spoofed.  You can trust the information in Received headers only
as far back as the last mail host you trust to report
information accurately.  If there is a spammer deliberately
deciding to deceive you, anything earlier is likely to be trash,
not because the wrong information is in the Whois tables, but
because the data in the Received fields is being made up (either
at random or in the hope of pinning the spam on someone else).
And, again, formalizing  the trust relationships through
contracts or licenses would help a great deal, and might be the
only solution, but those options carry a high price in terms of
generally accessibility and usability of email.

regards,
    john

p.s.

> Asking questions, presenting possible solutions, and learning
> from mistakes is how we get solutions.

And learning what has been done and discussed in the past,
before asking your questions of thousands of people, is usually
considered, at least, polite.

---

From: jfcm <[email-masked]>
Date: Mon, 13 Jan 2003 17:23:44 +0100

Dear John,
I am afraid that at this stage (e-mail + 40 or so years) telling someone to 
read the archives has no meaning. And telling him to post if he has a 
_new_idea either.

Could we not think of an FPS (frequently proposed solutions) where each 
defeated "solutions" would be listed and quickly discussed. There would be 
two good reasons:

1. to provide a true list of what has been proposed. It would save time to 
all and provide a good negative check list for those with an idea. At least 
it would be new to the FPS: it would be added or used.

2. very often the roots of the true solution is something which has been 
half thought and overlooked. Or something triggered in someone's mind by 
another idea.

jfc

PS. From what you quote, you seem to consider that SPAM=spoofing? Are there 
statistics and trends about that?

On 00:22 08/01/03, John C Klensin said:

>---
>Incoming mail is certified Virus Free.
>Checked by AVG anti-virus system (http://www.grisoft.com).
>Version: 6.0.427 / Virus Database: 240 - Release Date: 06/12/02

---

From: John C Klensin <[email-masked]>
Date: Mon, 13 Jan 2003 15:06:07 -0500

--On Monday, 13 January, 2003 17:23 +0100 jfcm <[email-masked]>
wrote:

> Dear John,
> I am afraid that at this stage (e-mail + 40 or so years)
> telling someone to read the archives has no meaning. And
> telling him to post if he has a _new_idea either.

You are entitled to your opinion.  I was only trying to suggest
that people who come to the IETF list, and propose the same old,
failed, ideas as if they had just received a relevation from on
high are likely to get some resistance... and to explain that
resistance.

> Could we not think of an FPS (frequently proposed solutions)
> where each defeated "solutions" would be listed and quickly
> discussed. There would be two good reasons:
> 
> 1. to provide a true list of what has been proposed. It would
> save time to all and provide a good negative check list for
> those with an idea. At least it would be new to the FPS: it
> would be added or used.
> 
> 2. very often the roots of the true solution is something
> which has been half thought and overlooked. Or something
> triggered in someone's mind by another idea.

Variations on this idea have been proposed to the IESG and IAB
several times, and have not gone anywhere.  I'll leave
explanations as to why to someone else, but  at a minimum, there
has been a shortage of volunteers to maintain a "dumb ideas
archive" (I know, that isn't quite what you said) and a shortage
of entities willing to shield such volunteers from liability.

> PS. From what you quote, you seem to consider that
> SPAM=spoofing? Are there statistics and trends about that?

There is certainly non-spoofed spam, including the many
materials that claim one has subscribed to an opt-in list or and
others that claim they are conformant to some law which never
passed.  I don't have any statistics that go beyond the
anecdotal.  But, if you look at the mass e-mailing software
packages that are frequently advertised (not exclusively by
spam), most of very proud of their capabilities to hide actual
message origins and to use the facilities of others as relaying
in supposedly-undetectable ways.  Similarly, as Doug and others
have observed, spam often comes with headers that are
sufficiently spoofed to make addresses and other data useless.
I assume --but cannot prove-- that all of these symptoms are
indications that spammers know that messages that use consistent
and accurate origin information are easily filtered out and
discarded and that most ISPs have terms and conditions of
service that prohibits using their facilities to spam.

regards,
     john

---

From: jfcm <[email-masked]>
Date: Mon, 13 Jan 2003 22:57:12 +0100

>high are likely to get some resistance... and to explain that

I know and I agree. My point is 40 years of reading is somewhat long and 
uncertain.

>of entities willing to shield such volunteers from liability.

hmmmm. Let take it another way. Would it help if we set-up
a place where each proposition could be registered in a few lines
by the proposer or dead ends documented by old-timers?

Instead of flaming proposers, they could just advise them to
register their propositions first. The flaming would still be here
but the register would be built.

As to know if it would be a dumb or a too brillant ideas archive :-)
jfc

---

From: Dave Crocker <[email-masked]>
Date: Wed, 15 Jan 2003 18:17:24 -0800

John,

Before someone makes suggestions about the magic bullet that will solve
spam problems, they should at least familiarize themselves with the
rather interesting range of startup company approaches to handling the
problem.  Everything ranging from keyword filtering by a commercial
version of spamassassin, to patenting a haiku.

And they should become familiar with the public policy and politics
debates on the topic.

This is a multifaceted problem, including the minor fact that people's
definition of "spam" is highly variable.  At this stage it appears clear
that no single magic bullet is possible and that we should start viewing
spam the way we view roaches.  We don't like them.  They are bad.  We do
a range of things to get rid of them.  It all helps.  But we do not
eliminate them.  We simply reduce them to a tolerable level.

d/

Tuesday, January 7, 2003, 3:22:06 PM, you wrote:
John> Almost all of the measures you have suggested have serious
John> side-effects or critical prerequisites.  In the last analysis,
John> most of us would rather put up with a little spam than pay the
John> prices involved.  Others are sufficiently fed up with spam that
John> they are willing to consider some very radical changes to how we
John> use email.  But, regardless of how that comes out, the decisions
John> have been fairly explicit: people have thought of your
John> suggestions, and others, and their impact, and have made fairly
John> explicit decisions about preferences.  My comment about X.400 of
John> a few weeks ago was intended to address those issues, but
John> apparently made a reference too far in the past, or too subtle,
John> for some of the people who have been participating in the
John> discussion.

d/
-- 
 Dave <mailto:[email-masked]>
 Brandenburg InternetWorking <http://www.brandenburg.com>
 t +1.408.246.8253; f +1.408.850.1850

---

From: John C Klensin <[email-masked]>
Date: Thu, 16 Jan 2003 08:34:16 -0500

--On Wednesday, 15 January, 2003 18:17 -0800 Dave Crocker
<[email-masked]> wrote:

> John,
> 
> Before someone makes suggestions about the magic bullet that
> will solve spam problems, they should at least familiarize
> themselves with the rather interesting range of startup
> company approaches to handling the problem.  Everything
> ranging from keyword filtering by a commercial version of
> spamassassin, to patenting a haiku.
> 
> And they should become familiar with the public policy and
> politics debates on the topic.
> 
> This is a multifaceted problem, including the minor fact that
> people's definition of "spam" is highly variable.  At this
> stage it appears clear that no single magic bullet is possible
> and that we should start viewing spam the way we view roaches.
> We don't like them.  They are bad.  We do a range of things to
> get rid of them.  It all helps.  But we do not eliminate them.
> We simply reduce them to a tolerable level.

Is there some part of this with which you assume I disagree?

     john

---

From: jfcm <[email-masked]>
Date: Thu, 16 Jan 2003 20:45:44 +0100

Dave, John,
at least in the case of spamming, it seems there is an agreement on the 
interest of cataloging the internet engineering frequently proposed 
solutions, to get a complet picture of the various existing and dropped 
propositions. This might both help not to repeat the same propositions and, 
may be, to discover usefull variations or, anyway, to help the thinking.

I understand that the difficulties would be:

- this could be perceived as a way to laugh at propositions. This is not 
the idea, and it can be adressed in having the author registering himself 
his suggestion, with a field reserved to quote the corresponding drafts 
and/or resulting RFC(s). It would help to quote references in drafts.

- this has neither to be verbose nor too terse. I would suggest a character 
limit and a link to a position page or a site (case of a commercial 
proposition) of the author.

- this should help debates. The author should be able to update his text 
and links.

- one needs the resources and staff. I am ready with anyone interested to 
start working on this if it may help. I made registered "iefps.org" by 
world@wide to that end (so we may freely discuss it). May be could ut be 
understood as "internet engineering first proposed suggestions" to make it 
some kind of historical repository (showing this is a positive service). 
Also a way to pay our common due to the initial good idea's proposers (like 
for the Cluster or the Catenet issues?). To do that we could think of a 
"ticket" system, the author may register and update further on (but not 
delete to keep it serious). It would ask for some classifications or 
keywords permitting to sort them by themes?

jfc

---

From: jfcm <[email-masked]>
Date: Mon, 13 Jan 2003 17:23:44 +0100

Dear John,

jfc

---

From: Jaap Akkerhuis <[email-masked]>
Date: Fri, 17 Jan 2003 16:04:53 +0100

Note that there is for quite some time a RIPE WG which is looking
into spam issues. See https://www.ripe.net/ripe/wg/anti-spam/index.html
for details.

	jaap