Subject: [Asrg] 0. General - Define Spam

From: "Jon Kyme" <[email-masked]>
Date: Sun, 04 Jan 2004 11:40:46 +0000

Well, I might as well start this before someone else does...

The new charter seems to define spam as "problematic email".
I'm not sure what this means. It seems to me that, given the vagueness of
the charter on this point, we'll need a definition before any progress can
be made. There's the "unwanted email messages" idea (see
http://www.irtf.org/asrg/ )
Is that good enough? ;-)

--

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: "Denny Figuerres" <[email-masked]>
Date: Sun, 4 Jan 2004 12:53:05 -0500

I was just on a weblog talking about HTML in email.
As I wrote a post I felt it should get read here also:

for the interested folks....

the real problems with HTML email are as follows:

1) scripts in the email (JavaScript and such)
2) image links in the email that may tell the server you have read the
email.
3) image links that look like
SRC="http://www.domain.com/CGI-BIN/image.pl=dgfd36546"
that windup logging your action and may also be a way for spyware and such.

I'd like to see an open standard for a ML based on html but it would ban the
above items, no external links to any image, program, style sheet, script
file etc...
no client side script tags.
no "Malformed" HTML tags as used in spam to hide the message from filters!!

if we could have a standard like that, it could help make HTML format safe
for users and less of a problem for the folks who currently don't want it.
infact I bet 90% of the "No HTML email" policy's would go away given the
rules I just sugjested. 
Basicaly a minimal-html subset that would allow css inline's and attached
in-the-email images but no outside links.
So the content would "stand alone" and would not have any script.

This could be a new mime-type standard that would allow current html
rendering logic to work.

A minimal set of software changes in email clients to verify the the
decalred mime-type was compling with the rules.

No breakage of SMTP,POP3 etc....

Just an idea.

---

From: "Vinny" <[email-masked]>
Date: Sun, 4 Jan 2004 14:35:45 -0600

Yeah, I was thinking much the same thing recently. The real problem though,
is not the fact that we have HTML mail, but that images embedded in HTML
mail are sourced from a web site, and that the web site catalogs what images
are accessed, therefore showing which email addresses are active. And your
suggestion that we have no images in email... well, can you imagine what the
public outcry would be against that? Joe Sixpack wants his images, and so do
various e-tailors like barnes and noble, amazon, etc.

A better idea would probably be to include the images inside the email as an
attachment, as you suggested.

I was thinking though, what about a whole new page description language?
HTML has always seemed a bit wasteful to me, since you have to go explicitly
set values like SRC, HREF, etc. What about something like <b14i>NARF</>,
where the first character in the opening tag represents the color (bLUE),
the next two the text size (14), the next character the font weight
(iTALIC)?

Interesting line of thinking. I'd love to see this thread end up to becoming
a project.....

-Vinny

----- Original Message ----- 
From: "Denny Figuerres" <[email-masked]>
To: "'ASRG'" <[email-masked]>
Sent: Sunday, January 04, 2004 11:53 AM
Subject: RE: [Asrg] 0. General - idea to reduce problems with message
content.

---

From: John Levine <[email-masked]>
Date: 5 Jan 2004 02:46:15 -0000

>The new charter seems to define spam as "problematic email".

Right.

An exact definition spam is a political topic, not a technical one,
and is not within the purview of ASRG.  If you look at all the halfway
plausible definitions of spam, such as UBE, UCE, or mail that users
complain to their ISPs about, they all describe about 95% the same
mail, so for most purposes any definition will do.

For any proposed anti-spam topic or technique, it would be a good idea
to describe what it deals with.  For the various LMAP proposals, for
example, it appears to me to be "mail sent from hosts other than ones
authorized by the owner of the domain in the envelope return address."

The question of a precise definition of spam has been beaten to death
at such length in so many places that I'm surprised to see it coming
up again.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
http://www.taugh.com

---

From: "Jon Kyme" <[email-masked]>
Date: Mon, 05 Jan 2004 12:31:32 +0000

>An exact definition spam is a political topic, not a technical one,

The reformulation of the problem as one of consent helped clarify 
views on this "social" (rather than "political" ?) ill.

>and is not within the purview of ASRG.  

Well, bless you too.

>If you look at all the halfway

Any definition? Right.

>authorized by the owner of the domain in the envelope return address."

This might seem like a "good thing" - but of course this means that there
is no way of comparing proposals meaningfully. e.g. Scheme (a) will
probably be (very close to) 100% effective at stopping the set of email
messages that the creator states it is targeted at. Unless the creator is a
complete idiot. Scheme (b) is likewise 100% effective for the (overlapping
but not identical) set it is designed for. How does this help anyone?

>at such length in so many places that I'm surprised to see it coming

I'm guessing you're missing the point on purpose. Or maybe you haven't been
following this list. But just to clarify - the consensus seems to be that
spam (as we call it) is "supported" in the MTS because of issues of access,
authentication, authorisation, anonymity, accountability and lack of
recipient "power". These *are* technically approachable issues (as well as
being often economic and contractual). These can be well-defined.
"spam" is not an entity in this technical domain (except as a "class" of
messages) which is why the group shouldn't depend on it.

*You* have rechartered the group to work on the poorly defined "spam"
problem. Previously, the group worked on issues raised by communication
characterised by (lack of) consent.

Furthermore, the group is now chartered to focus on "approaches that can be
defined, deployed and used in the near term". Research Groups are usually
focussed and long term, though short lived "task force" like Research
Groups are possible (I'm quoting). Is the group to be one of the latter?

I suspect that this rechartering is a "political" issue, perhaps there's
pressure to produce "appropriate" results in a predetermined timescale.
 
Regards,
JK

--

---

From: "Alan DeKok" <[email-masked]>
Date: Mon, 05 Jan 2004 11:14:59 -0500

> but not identical) set it is designed for. How does this help anyone?

  Because the two different schemes have *clearly* defined what part
of the spam problem they are trying to solve.  They can therefore be
compared on the level of "which part of the problem they solve", and
not just on "how they solve the problem".

  If we had one "perfect" definition of spam, then we would have two
ways of comparing anti-spam schemes:

  1) solely on their technical merits

     This is difficult, as the methods will be doing wildly different
     things.

  2) On the part of the spam problem they address, AND on #1

     For this, we will still need a separate definition of the part of
     the problem the scheme addresses, independent of any "perfect"
     definition of spam.

> characterised by (lack of) consent.

  I must admit that my preference would be to have the charter discuss
consent.  We can then come up with a "perfect" definition of spam
which fits within the consent model:

  SPAM: For a particular system, any message locally determined to be spam.

  Nice, circular, and nearly useless.  But it does focus the solution
where it should belong: *I* determine what is spam on my system, and
any system which is set up to permit *other* people to make that
decision on my behalf should be explicitly outside of the group.

> defined, deployed and used in the near term".

  Given the blatant opposition to getting anything done, I'm inclined
to believe that the SMTP-based email system will collapse before
people are willing to do anything.

  I've seen a ~10x increase in spam to my systems in the past two
months.  It's *ridiculous* that there's so much opposition to doing
anything.  Of *course* any anti-spam system will be short-term, and
won't solve the problem.  But it should give us the breathing room to
work on a more complete solution.

  i.e. I'd rather not have my doctor say "You've got terminal cancer,
but we're not going to treat it.  It's easier for us to wait until
you're dead, before we operate.  We think that by then, we'll have a
better treatment available."

  That's blatant incompetence and negligence.  But on the anti-spam
front, it's accepted practice.

  Alan DeKok.

---

From: Dave Crocker <[email-masked]>
Date: Mon, 5 Jan 2004 08:26:52 -0800

Jon,

>  views on this "social" (rather than "political" ?) ill.

Indeed.  Notice how productive things have been, spending lots of time 
debating definitions (repeatedly), lots of time making ad hominem 
attacks, and no success at getting supportive technical work done.

>  probably be (very close to) 100% effective at stopping the set of 
email

Careful use of the evaluation points suggested in the Technical 
Considerations draft will show such things as adoption difficulties and 
ease with which spammers can 'route around' the proposed mechanism.  
This does not require a comparison with other proposal; it requires 
conceptual and specification quality that can be evaluated on its own.

Development of strong community support obviates the need for doing 
comparisons and artificially declaring choices among them.

>  the consensus seems to be 

1. This list has yet to show anything that satisfies the historical use 
of the term "rough consensus".

2. This list has primarily succeeded at convincing active anti-spam 
workers to spend their time elsewhere; as such a consensus in the 
current group is what statisticians politely call "not representative of 
the larger population".

>  problem.

They have rechartered the group to work on technical specifications, 
rather than endless debate about definitions.

>  Research Groups are usually
>  focussed and long term, 

1. "research" is a term used with variable meaning, in terms of 
practicality and timeframe.

2.  "long term" is a term used with variable meaning.

2. There is a difference between "long term" and "forever".  The rate of 
progress in this group encourages the latter.  

>  pressure to produce "appropriate" results in a predetermined 
timescale.

Absolutely correct.

We need effective mechanisms, defined, deployed and used, before spam 
destroys email.  We are already under crisis pressure.

d/

--
Dave Crocker <dcrocker-at-brandenburg-dot-com>
Brandenburg InternetWorking <http://brandenburg.com>

---

From: Art Pollard <[email-masked]>
Date: Mon, 05 Jan 2004 09:58:47 -0700

At the risk of seeing this whole discussion brought up yet again ....
(Though I guess it already has.)

Rather than trying to come up with the "perfect" definition of spam, we 
simply need to define it / use it in such a way that it will provide the 
most immediate benefit.  Of course, the definition: "Spam is whatever the 
end-user defines it to be" is probably the most accurate in many people's 
minds and in particular our own.  However, it is not useful as was 
mentioned in an earlier post.

The immediate problem we are all facing is the sending of "unwanted 
automated bulk e-mail." (Or should I say the receipt of it.)  And that is 
the definition we should use for the time being.  Once this problem has 
been solved or nearly so, then perhaps we can think of ways to deal with 
"unwanted e-mail", "unwanted bulk e-mail" (that is sent out by hand by 
somebody in a third world country hired for their cheap labor rates), or 
other definition.  In the meantime, the bulk (pun) of the problem will have 
been addressed and hopefully taken care of.  The mail volume will be 
significantly less and at that point, it makes sense to find ways to 
reassess our position and the problem.

-Art
-- 
Art Pollard
http://www.lextek.com/
Suppliers of High Performance Text Retrieval Engines.

---

From: "Jon Kyme" <[email-masked]>
Date: Mon, 05 Jan 2004 17:03:03 +0000

AD>
> work on a more complete solution.

On the free webmail system that I'm familiar with, the "spam" problem has
been largely solved from the *user* perpective by simply using a couple
"low-power" and fairly conservative BLs
(currently stopping 31% of incoming mail)  followed up by SA based
filtering (37% to to a SPAM folder with automatic ageing out). a simple
whitelisting facility is also provided. Clearly, the provider is paying for
this with CPU and bandwidth, but both users and management seem quite happy

My employers use a broadly similar system on the corporate email system.
Again, usrs and mgmt seem happy.

Incidentally, on the webmail there's also a certain amount of egress
control. So it's thought that the potential for abuse is fairly well
controlled.

Now IS the breathing space. A complete solution (if such a thing is
possible) needs a problem definition. You know that I like the consent idea
- NOT because it gives us a "a 'perfect' definition of spam ... For a
particular system, any message locally determined to be spam.", but because
it enables us to work *without* a spam definition. Instead we have a thing
called consent which can be represented, transported, queried etc. We can
evaluate systems in terms of how well they support consent. We can worry
about edge/middle issues, anonymity and so on.

How to do all this - that's suitable matter for research - this may be the
long term way forward. This is not to say that all the work being done on
sender verification / content based filtering etc isn't worthy, of course
it is - that's what has given (is giving) us this breathing space.
 
Quick fixes have their place - I'm not convinced that place is a RG.

--

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 05 Jan 2004 15:21:44 -0500

>>work on a more complete solution.
...
> it is - that's what has given (is giving) us this breathing space.

We have no problems with researching consent as long as there is enough 
people willing to do so. However, we want to refocus the RG away from 
that, in order to be able to consider short and medium term solutions as 
well.

So far research into consent has not gathered enough interest in the RG 
or outside of it, to move it forward in any way. Therefore, we would 
like to refocus and also look at more immediate solutions which have 
enough interest in them to be researched and implemented.

Yakov
-------
Yakov Shafranovich / asrg <at> shaftek.org
SolidMatrix Technologies, Inc. / research <at> solidmatrix.com
"Power tends to corrupt, and absolute power corrupts absolutely" (Lord 
Acton)
-------

---

From: "Jon Kyme" <[email-masked]>
Date: Mon, 05 Jan 2004 20:45:42 +0000

> enough interest in them to be researched and implemented.

OK, I'll be quiet now.

--

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 05 Jan 2004 15:57:03 -0500

>>enough interest in them to be researched and implemented.

As I stated before if enough interest is gathered in researching consent 
long-term, we can form a subgroup for that. So far, there has not been 
enough interested and motivated people for that. So keep your ears open...

Yakov
-------
Yakov Shafranovich / asrg <at> shaftek.org
SolidMatrix Technologies, Inc. / research <at> solidmatrix.com
  "Why are both drug addicts and computer aficionados both called 
users?" (Clifford Stoll)
-------

---

From: Markus Stumpf <[email-masked]>
Date: Thu, 8 Jan 2004 02:27:33 +0100

> and is not within the purview of ASRG.

I have read a article in a message board (german) and the author made the
argument that spam is also a problem of context:
   http://www.heise.de/security/news/foren/go.shtml?read=1&msg_id=4831635&forum_id=51300
(I'll try do do a rough translation/interpretion of the examples).

If I write a sentence even in this forum (e.g. in the sig)
    "it is cold out there so you should buy and wear warm clothes"
it is neither advertising nor spam. It's simply a statement related to
the current weather situation for most of the subscribers in the
"winter" zone of this planet. Some may even interpret it as a well
meaning hint.

If the sender is <insert name of company producing winter jackets>
the context changes caused by the sender of the message and a lot of
people will make a relation to "sender is producing winter jackets"
and the statement will be seen in a completely different light and
some will interpret it as advertising.

It also depends on the contents of the message. Sending this
statement to a user in Alaska will probably produce a different
reaction (even in terms of nuissance) than sending it to someone
living near the equator (who by chance doesn't even know what "cold"
means ;-))))

	\Maex

-- 
SpaceNet AG            | Joseph-Dollinger-Bogen 14 | Fon: +49 (89) 32356-0
Research & Development |       D-80807 Muenchen    | Fax: +49 (89) 32356-299
"The security, stability and reliability of a computer system is reciprocally
 proportional to the amount of vacuity between the ears of the admin"