Subject: Re: [Asrg] Introduction and another idea

From: Yakov Shafranovich <[email-masked]>
Date: Thu, 19 Jun 2003 14:11:16 -0400

At 04:53 PM 6/15/2003 -0500, [email-masked] wrote:
>[..]
>During the discussion it struck me that a better approach to the problem 
>is the
>following:
>
>A recipient should be able to create a specific-permission "whitelist" which
>lists the senders (by E-mail address) to which they wish to assign "special"
>privileges.  Senders who are not on the whitelist would only be allowed to 
>send
>that recipient plain, unencoded, ASCII text E-mails without HTML, scripting,
>encoding, or any kind of attachments.
>[..]
>The whitelist would be maintained either at the recipient's mail server, or
>local ISP's mail server, or else (for their own domain) at the domain
>provider/mail forwarder... probably, as soon as the mail resolves to the
>addressee domain.

Maintaining white lists for users in places other than their computers 
raises privacy issues as mentioned many times prior.

>You'll note that this scheme does not provide for the blocking (at this 
>level,
>anyhow) of plain ASCII text spams.  True.  More on that later.
>
>But the great majority of spams use a variety of tricks to get past content
>filters, and to deceive recipients.  A *large* subset of those tricks are 
>based
>on HTML:
>
>   1)  HTML comments (and bogus HTML tags) to break up and obscure mail 
> content,
>and to confuse content filters;

HTML email is also used by many companies for legitimate purposes.

>   2)  graphic-mode-text (likewise);

Same comment as above.

>   3)  links which purport to be one thing but where the actual hyperlink 
> in fact
>(and usually invisibly) points somewhere else;

Many spammers own their own domains and will not lose out from this.

>   4)  scripting where the message displayed only can be viewed as a 
> result of
>the computational process, again to make things difficult for content filters.
>
>Attachments (especially in unsolicited E-mails) tend to frequently contain
>viruses, worms, "background music" that's actually a PIF or EXE file, and 
>things
>of that sort.  Getting attachments from someone who doesn't ordinarily send
>those is a warning sign that it might well be malicious.

Not all attachments are bad.

>[..]
>By enabling a user to simply t-can any unexpected HTML-burdened (or
>attachment-carrying) incoming message (and ideally as soon as it got to their
>domain provider or ISP), spammers would be denied many of their most 
>cherished
>and valuable tricks.  Content filters would be far more useful and 
>efficient.
>And much more of the remaining unsolicited spam that WOULD still be sent 
>would
>be sent in plain ASCII text (knowing that sending unsolicited HTML E-mail was
>the kiss of death...) meaning it would reduce wasted bandwidth for such
>remaining spam mail net-wide by at least a factor of three to five.
>[..]

Thus proposal addresses many of the tricks that are used by spammers TODAY. 
As Vernon and many others mentioned on the list before spammers change very 
quickly and can adapt their practices very quickly as well. This scheme 
would be just a band aid to block some forms of spam until spammers will 
figure out a way around it just like many other proposals.. Additionally, 
many of the "Nigerian Scams" arrive as plain ASCII emails already and this 
scheme will not stop them.

Another concern with this sceme is the fact that email will go back to the 
dark ages with no support for attachments and no HTML support. This may 
increase people's use for email which is already under attack by spammers. 
Is the medicine as bitter as the problem?

Also, blocking base64 encoding would block email schemes where digital 
signatures are used.

Yakov

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Vernon Schryver <[email-masked]>
Date: Thu, 19 Jun 2003 12:41:46 -0600 (MDT)

> From: Yakov Shafranovich <[email-masked]>

> ...

That is true only in absolute numbers, because the Internet is so
large that even tiny minorities can involve big numbers.  The
overwhelmingly vast majority of the use of HTML is at best a waste
of bandwidth and CPU cycles.  There is far more use of HTML mail
to violate privacy with such as "web bugs" than legitimate use.

We will know that Microsoft really cares about spam not when when
they file more lawsuits, but when they change the default configuration
of Internet Explorer to avoid generating HTML mail, when they chagne
Internet Explorer to not just assume that plaintext mail that
contains strings that look like HTML is HTML, and when they make
it easy for Hotmail and MSN users to reject HTML mail.

HTML will always be a fertile ground for privacy violations and other
nastiness.  If not for HTML mail, users would be less conditioned to
accept worms and viruses and would instead be naturally suspicious of
"active content."

I'm sure there are legitimate uses of HTML mail, but I cannot remember
ever receiving one.  I bet that you have never received HTML mail that
would not have been better in plaintext, perhaps with a URL pointing
to a web page.  (Note that modern browsers present URLs in plaintext
as "clickable links.")

> ...

There is a difference between having no support for fancy features and
using those fancy features everywhere and where they are a positive
hazard to computer security and personal privacy.

> Also, blocking base64 encoding would block email schemes where digital 

That is entirely mistaken.  You might want to ensure that sender and
recipient agree on whether to sign the encoded or plaintext versions
of the message, but that's all.

Vernon Schryver    [email-masked]

---

From: "Bob Wyman" <[email-masked]>
Date: Thu, 19 Jun 2003 14:52:11 -0400

Yakov Shafranovich wrote:
> Maintaining white lists for users in places other 
> than their computers raises privacy issues as 
> mentioned many times prior.
	As has also been mentioned many times in the past, the use of
"virtual" or "implicit" whitelists by means of using single-user
addresses or passing authentication tokens as part of messages can give
the effect of whitelists while reducing significantly the privacy
issues. (since the whitelist is distributed, it is harder to
reconstruct.)
	When discussing "whitelisting" we should not confuse the
mechanism with the concept. The concept is that some subset of the
entire universe of senders is given preferred access to an inbox. This
can either be by checking against an explicit whitelist which is
represented as a monolithic list of senders and privledges or it can be
done algorithmically by checking properties of the sent message (i.e.
tokens in headers or in addresses) to determine if the sender is on an
"implicit" or "virtual" whitelist.
	The privacy issues with whitelists only apply to some of the
mechanisms for constructing and/or maintaining them.

		bob wyman

---

From: "Bob Wyman" <[email-masked]>
Date: Thu, 19 Jun 2003 15:04:23 -0400

Vernon Schryver wrote:
> The overwhelmingly vast majority of the use of HTML is 
> at best a waste of bandwidth and CPU cycles.  
	This is, at best, a representation of a personal, subjective
opinion. The reality is that there is great demand for "rich text"
messaging and there always has been. There are many clear indicators
that large numbers of people like to both send and receive HTML
formatted messages. This demand is not new.
	As far back as 1981, when I first started selling one of the
first "corporate" email systems, Digital's ALL-IN-1 office, customers
were asking for and often demanding "rich text" or "high quality" email
messaging capabilities. If anything, the demand for such a capability
has increased over time. 
	It is no more reasonable to demand that the world accept that
email should not contain rendering information than it would be for some
religious sect to demand that we not "pollute" the walls of our homes
with art. Our challenge should be to figure out how to give the users
what they want without exposing them to unnecessary risks. If the
problem is a difficult one to solve, it isn't because what the users
want is some that is not "legitimage," rather, it simply means that
we're not being creative enough in defining solutions. 

> There is far more use of HTML mail to violate privacy 
> with such as "web bugs" than legitimate use.
	I don't believe this to be true. I get loads of "legitimate"
HTML mail every day. In any case, it doesn't matter. The issue is how to
design a system that works, not how to restrict legitimate users from
accomplishing their goals or seeing "pretty" email messages.

		bob wyman

---

From: Vernon Schryver <[email-masked]>
Date: Thu, 19 Jun 2003 13:27:39 -0600 (MDT)

> From: "Bob Wyman" <[email-masked]>

> formatted messages. This demand is not new.

I find your claim even more of personal, subjectve, and inaccurate.

As you say, there has been a great and legitimate demand for "rich
text messaging," but the vast, overwhelming majority of HTML email
uses nothing that might be called "rich text messaging," unless you
equate soft line breaks with rich text.  The vast majority of users
do not use any sort of formating including bold and italics in their
personal mail.  They can barely manage to turn on spell checking.

Essentially all of that great demand for rich text is from sales
professionals of one sort or another and other people writing formal
message, and they are generally unable to manage more than what might
be called Powerpoint-HTML.

> messaging capabilities. If anything, the demand for such a capability

Of course, but how much of that "high quality email" is used except
in various kinds of sales presentations?  The overwhelmingly vast
majority of email is informal.  It makes as much sense to say that
much email should use HTML as to claim the same for SMS....don't tell
me that some sort of 3G cellphone gateway is going to use HTML for
SMS because I'll believe you, and because it would support my point.

> with art. 

That's nice, and unrelated to what I wrote.  I didn't say that Hotmail
accounts should reject HTML by default but only that Hotmail users
should be able to reject HTML mail.  I also didn't say that IE and
Netscape should not be able to generate HTML mail, but only that they
should not generate it by default.  Both were intentional.

>           Our challenge should be to figure out how to give the users

I carefully did not say that all HTML is illegitimate.  So why are
you implying that I did?  Do you have some vested interest in HTML
mail software?  As I said, some HTML mail is legitimate and because
of the size of the Internet, that tiny minority is a lot of messages.
However, that tiny minority of legitimate HTML mail has nothing to do
with the vastly larger bazillions of messages from end users that are
in HTML only because their MUAs use HTML by default.

Then there are the bazillions of spam in HTML.

> accomplishing their goals or seeing "pretty" email messages.

I'm happy you're happy with your legitimate HTML mail.  However, I
stand by my claim by pointing to the vast quantity of spam and of
legitimate, (because explicitly solicited) marketing mail that contains
web bugs.  How many marketing organizations that have resisted the
sales calls from the mail coverage tracking vendors has not implemented
their own tracking systems?

---

From: Yakov Shafranovich <[email-masked]>
Date: Thu, 19 Jun 2003 15:32:15 -0400

>Then there are the bazillions of spam in HTML.

Then the question is do we sacrifice this minority of legit HTML messages 
in order to kill spam.

---

From: Vernon Schryver <[email-masked]>
Date: Thu, 19 Jun 2003 13:47:26 -0600 (MDT)

> Then the question is do we sacrifice this minority of legit HTML messages 

Of course not.  Who suggested anything of the sort?

We still have legitimate SMTP relays.  Suggestions that all SMTP relays
be outlawed or sacrificed come from only kooks and those who are so
completely ignorant of the system that they don't know about MX
secondarys and "smart hosts."

Making HTML email not be the default sacrafices nothing more than
changing the default in sendmail, which was not a sacrafice at all.

---

From: Benjamin Geer <[email-masked]>
Date: Thu, 19 Jun 2003 20:57:43 +0100

On Thu, Jun 19, 2003 at 12:41:46PM -0600, Vernon Schryver wrote:
> ever receiving one.

I maintain a number of mailing lists which the members of various
(non-technical) organizations use to communicate with each other.  The
volume of legitimate HTML email on these lists is quite high.

Programmers and sysadmins, and people who started using email before MIME
was introduced, tend to dislike HTML email.  Other people see the ability to
use boldface and italics as normal features of any text-editing environment
(which they consider their MUA to be).  They would find the idea of
suppressing those features simply bizarre.  When writing a document that is
going to be printed, whether we use Microsoft Word or LaTeX, we can put the
title of a book in italics; people have come to expect that the same is true
when they write email.

As for attachments, nowadays, many people have digital cameras and
high-speed Internet connections, and like to send photos to their friends
via email. It is no use telling these people that 'sending large attachments
is poor netiquette'; they and their friends have the requisite bandwidth,
and it doesn't bother them.

My own view is that these issues are red herrings.  Spam is spam whether it
consists of an ASCII text file or a JPEG.  Effective anti-spam measures
should stop spam regardless of its format.

Ben

---

From: Vernon Schryver <[email-masked]>
Date: Thu, 19 Jun 2003 14:21:45 -0600 (MDT)

> From: Benjamin Geer <[email-masked]>

> volume of legitimate HTML email on these lists is quite high.

Is it high because those people use bold, italic, and so forth or
because their MUAs send in HTML by default and they are not savvy
enough to fix that problem?

> was introduced, tend to dislike HTML email. 

There is some truth to that, but it is overstated.

>                                              Other people see the ability to
> (which they consider their MUA to be). 

That is also true, but overstated.  The vast majority of email is too
informal to have spell checking except where it is automatic (and I
doubt most users bother to turn it on).  People can almost never
(statistically) be bothered to notice typos confounding "you're" with
"your", and use bold and italics even less often.  However, that
implies nothing about not having reasonable text formating (not
"editing") facilities when you want them.

>                                         They would find the idea of

What I find bizzare is the persistent equating of turning off HTML by
default with banning HTML.  Those responses make me wonder if I've
not been paying enough attention.  Is HTML mail being summarily rejected
by ISPs and commercial mail gateways?

Given facts of life including spam and web bugs, why would that be
worse than filtering mail with large attachments due to facts of life
including worms?

> and it doesn't bother them.

That misrepresents the advice.  It is in fact poor netiquette to send
large attachments without knowing your targets want and can handle
them.  Once you know, there's no problem.

Are you advocating a right to send HTML or MByte attachments to perfect
strangers?  Why didn't you send your message in HTML, perhaps with a
picture of your significant other, house, new baby, car, or something
else close to your heart?

> should stop spam regardless of its format.

Yes, but equating not sending attachments without knowing your
correspondents want them with banning attachments or not using HTML to
send simply ASCII mail to banning HTML is a worse red herring.

---

From: Selby Hatch <[email-masked]>
Date: Thu, 19 Jun 2003 15:22:47 -0600

Yakov Shafranovich wrote:
> Then the question is do we sacrifice this minority of legit HTML 
> messages in order to kill spam.

I agree. We should not sacrifice any email types or formats to kill spam.

Rather than limit the types of email that we receive, we should state 
that we will only receive email from (1) a server that can be verified 
as the one from where the email originated and (2) an individual or 
entity who has received authorization from me to send me email, which 
authorization can be verified upon receipt.

By limiting the mime types, base64, html, etc., we limit a great deal of 
useful features.

---

From: Vernon Schryver <[email-masked]>
Date: Thu, 19 Jun 2003 15:52:11 -0600 (MDT)

> From: Selby Hatch <[email-masked]>

> authorization can be verified upon receipt.

That's a good policy for people for whom mail is only a recreation or
hobby.  It does not work for people whose bread and butter depends on
receiving mail from strangers. 

There are also problems with (1).  As stated it is impractical.  For
example, it outlaws smart hosts.  It also doesn't do anything about
the large quantity of spam that comes from spammers' own computers.
There are also major practical problems with notion of "trust."

Barry Shein's rant about criminal hijacking was only 53.2% right.
It ignored the fact that a lot of spam advertises web pages on
non-hijacked systems.  Since few if any ISPs don't outlaw spamvertising,
the reason spammers use proxies and relays can't be fears of having
accounts terminated.  Instead, they use criminal mechanisms to evade
IP address and domain name blacklists.

> By limiting the mime types, base64, html, etc., we limit a great deal of 

The notion of outlawing MIME, Base64, HTML, etc among consenting MTAs
and MUAs is red herring.  No one can seriously propose eliminating them.

However, you would cut down you spam load significantly if you rejected
all mail with "Content-type: text/html" SMTP or MIME entity headers
from strangers.  24672 or 73% of the 33807 spam in my 39 day rolling
log of what has been sent toward my traps and filters has that mark.
(It should be 40 days, but I had an attack of bad hands and eyes
this morning.)

---

From: Benjamin Geer <[email-masked]>
Date: Thu, 19 Jun 2003 22:55:18 +0100

> enough to fix that problem?

I'm not sure, but my impression is that there's a fair amount of both.

> default with banning HTML.

Fair enough.  But even if all MUAs turned off HTML by default, spammers
would continue to use HTML.

Getting back to Gordon Peterson's proposal, he suggests a mechanism which
would discourage the use of HTML and attachments, by forcing people to
explicitly give others permission to send them those types of messages.
This is meant to reduce the effectiveness of spam and address-harvesting
techniques, and increase the effectiveness of content filters.

Unfortunately, I think the vast majority of computer users have absolutely
no idea what HTML is.  As a sysadmin, I've tried to explain it to people,
and their eyes glaze over. They also have trouble grasping the notion of
'plain text', i.e. a format that doesn't support all the things they
normally think of as being available in 'text' (which for them means what's
available in Microsoft Word).  Once they understand the severe limitations
of this wonderful 'plain text' format which I'm encouraging them to use, the
first thing they want to know is how to make sure that they never use it.
They *like* the fact that when you reply to a message, Outlook puts the
quoted text in a different colour; they feel that it makes messages easier
to read.  To them, right angle-brackets look like something out of the Stone
Age.

Imagine Sally User, who is a customer service representative, or a
freelancer.  Her ISP starts offering the proposed permissions service.  She
has no idea what HTML is, but she gathers that it's something about reducing
spam, so she accepts the default permissions, which are quite restrictive.
Suddenly people who write to Sally find that their messages bounce, with an
automated reply saying: 'Please send me a plain-text message asking for
permission to send me HTML email or attachments.' These correspondents have
no idea what this means (remember, they don't know what HTML is, either),
and are put off by the fact that their email was rejected.  So they simply
take their business somewhere else, to another company or freelancer who
doesn't reject their email.  When Sally finds out, she's horrified, and
quickly sets her permissions to accept anything from anyone.

I suspect that most people, if they understood that restrictive permissions
would cause legitimate messages to bounce, would opt for giving full
permissions to all senders.

As for making content filtering more effective, it would be easy for content
filters to strip out all the HTML tags, comments, JavaScript, etc. from a
message before filtering.  No need to actually parse the HTML.  (I'd be
surprised if there weren't content filters that do this already.)

Finally, even if this proposal worked exactly as planned, why couldn't
spammers do just fine with ASCII?  The Nigerian 419 spammers and their
imitators have been using ASCII all along, and if they were a bit more
clever about their wording, it would be quite difficult to filter.

Again, I think this is all really beside the point.  We're here to come up
with a mechanism to stop unsolicited bulk email.  Gordon's proposal is
concerned with some superficial properties that much spam currently has, but
doesn't deal with the core issues: 'unsolicited' and 'bulk'.

Ben

---

From: Selby Hatch <[email-masked]>
Date: Thu, 19 Jun 2003 16:13:11 -0600

> receiving mail from strangers. 
(snip)
> from strangers.  

How can you have it both ways? You either receive email from strangers 
or you don't. Those who want to recieve mail from strangers can simply 
not block any email, or continue to use filters. Those who don't, should 
be able to require verifiable identification through some means.

If one's bread and butter dependes on receiving email from strangers, 
then they need an open mail box. Most (I believe a very high percentage) 
of email users don't depend on email from strangers for their living.

---

From: Vernon Schryver <[email-masked]>
Date: Thu, 19 Jun 2003 19:20:49 -0600 (MDT)

> I'm not sure, but my impression is that there's a fair amount of both.

Why don't you test that impression?   This is a question that can be
measure, and I think makes sense.  How much of that mail involves real
formatting?  If you choose 50 messsages at random, how many use HTML
and of those, how many use any real text formating?  (e.g. intentional
use of <em> or <b> as opposed to noise added by MUAs to all HTML
messages they create)

If you'll take bets, put my money on 80% or more using HTML and
less than 1% using any real text formating.

> would continue to use HTML.

You say that as if it were a bad thing.  On the contrary, if spammers
label their stuff as spam, it becomes harmless to all practical purposes
(screams of rage from spammer fighters not withstanding).

> techniques, and increase the effectiveness of content filters.

Yes.

> and their eyes glaze over. ....

Yes, but that cuts both ways.  I bet most users have no clue
how to tell IE or Netscape how to make a word bold or red.

As I test, I just now tried to get Netscape 7.02 on XP to make a word
bold in a mail message.  I cannot find any sign of any way to do such
a thing.  I assume there is a way, but that I can't find it suggests
that your Sally End User might not use it very often.  (My first blush
with computerized text formating predated my first encounter with the
Internet on the console of TIP-25 (DOCB) in 1972.  I also wrote one
of NBI's text processing systems, so I flatter myself in thinking I'm
not entirely adverse to fancy text.)

>                        ...     Once they understand the severe limitations

So turn off HTML until the the user makes some text bold or purple,
and then silently and automatically switch to HTML.

> to read.  To them, right angle-brackets look like something out of the Stone

You are confusing presentation with encoding.  There are MUAs that
recognize various > markings and colorize quoted text.  I think they're
garish and distracting, but there's no disputing tastes.

> permissions to all senders.

That is contradicts experience reported by people running retail ISPs.
Many ISPs report almost all of their customers are happy with single
digit false positive rates for 80% reductions in spam loads.  I think
80% is a modest improvement and that 100 times smaller false positives
rates of 0.1% are at the high end of the tolerable range, but I'm
probably not typical of retail ISP customers.

Notice also that most proposed or existing spam defenses that people
advocate have single digit false positive rates.   

As I keep saying there are those of us whose livelihoods involve
email and who won't tolerate 0.2% false positives and there are
the 100,000,000's of the rest of the Internet for whom not receiving
spam is more important than not hearing from a long lost friend.

> surprised if there weren't content filters that do this already.)

That's wrong in two ways.  What I consider reasonable filters (e.g.
the DCC) already strip HTML tags and so forth, but that turns out to
be insufficient.  For reasons you might guess if you knew that the
DCC is my hobby, I'd rather not say more to strangers.

> clever about their wording, it would be quite difficult to filter.

That's half right and half wrong.  Spammers need to appear as legitimate
as possible to their targets and to law enforcement.  Without HTML
tricks to hide various things, all of their stuff is in the open and
plainly visible to your Sally End Users.  Even your Sally probably
discards a Nigerian 419 spam after reading the first 5 words and she
may delete unread mail with obvious "hash busters" in subjects.

> doesn't deal with the core issues: 'unsolicited' and 'bulk'.

On the contrary, "solicited" is related to "consent," and Gordon's proposal
is all about "consent."

As for bulk, I know of exactly one idea with any hope of detecting that
aspect of incoming mail, and it is certainly not perfect, 100% accurate,
or invulnerable to spammer tricks.  That idea is that as a message
arrives or is about to be presented to you, your MTA or MUA should ask
a clearinghouse if substantially identical messages have been seen
elsewhere.  If the answer is "yes, 12345 of them," then your MTA or MUA
can know it is a "bulk" message check your list of approved senders or
whitelist.  For scaling, the clearinghouse should be distributed.  There
are other aspects involving scaling, privacy, security against various
attacks that need to be addressed.  When you've done that, you'll have
a direct competitor to the Distributed Checksum Clearinghouses or the DCC.
See http://www.dcc-servers.net/dcc/graphs/index.cgi

Again, I realize that the DCC is far from perfect.  That is one rason
why I've talked about it far less than other things here, and less
than other people have talked about their favorite mechanisms.

---

From: Benjamin Geer <[email-masked]>
Date: Fri, 20 Jun 2003 09:47:27 +0100

On Thu, Jun 19, 2003 at 07:20:49PM -0600, Vernon Schryver wrote:
> a thing.

Netscape 7 was written by people like us, who dislike HTML email, so they've
made it non-obvious.  But in the business world, nobody uses Netscape for
their email.  They use Outlook, which gives you a little toolbar that looks
exactly like your Microsoft Word formatting toolbar, with font size,
bold/italic/underline, colour, etc.

> digit false positive rates for 80% reductions in spam loads.

If 70% of people currently use HTML, as Kee Hinckley's numbers suggest, then
the false positive rate will be 70% under the proposed scheme.

Think of what would happen in those 70% of cases.  Instead of one message,
there would be at least three (the original HTML mail, the bounce message,
and the plain-text version), and at most five (the original HTML mail, the
bounce message, the request for permission, the acknowledgement of
permission, and then the original HTML mail again).  For a proposal that's
meant to decrease the volume of mail and avoid creating disruption, that's
not very satisfactory.

> spam is more important than not hearing from a long lost friend.

Mail from strangers isn't just from long-lost friends.  It's ubiquitous in
the corporate world.

The employees of a typical company communicate with employees of many other
companies: clients, suppliers, etc. Employees come and go, and different
projects involve different combinations of companies and different people,
so the set of people that a given individual may receive legitimate mail
from is constantly changing.  Consider a sales representative, who may be
contacted by a constantly-changing set of hundreds of people who work at
companies that she is negotiating sales with, or that may be interested in
her company's products.  As a sale progresses, more companies (accountants,
lawyers, etc.) are brought in.  Similarly for mergers and acquisitions.
That's a lot of people who would be annoyed by bounces.

> As for bulk [...] the Distributed Checksum Clearinghouses or the DCC.

Yes, I know, and to me, DCC seems much more pertinent to the task of this
group.

> than other people have talked about their favorite mechanisms.

So let's talk about it.  Could you summarize the problems of DCC (I know one
or two have been mentioned here already)?  Perhaps this group could give
some thought to whether those problems could have technical solutions.

Ben

---

From: Kee Hinckley <[email-masked]>
Date: Fri, 20 Jun 2003 09:10:54 -0400

>use of <em> or <b> as opposed to noise added by MUAs to all HTML

That turns out to be extremely difficult to detect.  In particular 
forwarded messages use lots of HTML formatting.  That and lots of 
other things, like links, image tags, font changes are *not* 
gratuitous formatting.  They make it easier for a user to pick out 
which parts of the message come from where.  Yes, we know there are 
other ways (Eudora creates vertical bars for quoted text without 
using HTML, and links show up as links, and attached images show up 
inline anyway).  But the fact of the matter is that HTML email is not 
just fluff--it's a component of the UI that makes it easier for many 
people to use email.  Whether they deliberately make a piece of text 
bold is not a determiner of whether they want their email formatted.
-- 
Kee Hinckley
http://www.messagefire.com/          Anti-Spam Service for your POP Account
http://commons.somewhere.com/buzz/   Writings on Technology and Society

I'm not sure which upsets me more: that people are so unwilling to accept
responsibility for their own actions, or that they are so eager to regulate
everyone else's.

---

From: Vernon Schryver <[email-masked]>
Date: Fri, 20 Jun 2003 07:46:19 -0600 (MDT)

> some thought to whether those problems could have technical solutions.

The main problem with the DCC and similar filters is obvious and has
been mentioned here.  It is fundamentally unfixable, and so I'd rather
not talk about it in public.

Tactics like the DCC should be assumed to be effective only against
"mainsleaze" or spam from the Fortune 50,000.  The current flood of
drugs, loansharking, and other frauds will eventually be controlled
by legislation that makes it a serious crime to not pay the "universal
mail access infrastructure free" of $0.001 to $0.10 per target.
Corporate spammers like Topica, DoubleClick, Microsoft, and Dell will
be happy to pay and reluctant to appear to be old style evil spammers
by being creative in their filter avoidance.

This list has not heard from the industry leaders in spam defenses
including Brightmail and Postini.  Even SpamAssassin and the so called
Bayesian filtering people hasn't had much to say.  I think most if
not all of them are (or were?) represented here.  Consider why they've
had little or nothing to say.  Their filters are all at least somewhat
effective, but could doubtless be improved.

I've noticed distinct, albeit superficial changes in spam in my traps
after remarks made here, so talking about limitations in existing
filters could be costly.  Committees are rarely effective in attacking
straight technical issues even when the participants understand the
technical issues.  Some ASRG contributors don't (or didn't) know the
difference between an SMTP error status and a DSN.  Many contributors
with better technical understanding seem to be limited in the spam
they see to a few dozen/day.  Thus, talk here about existing filters
risks a high cost for the fliters but small likelihood of benefit.

This group should concentrate on its charter, including defining the
problem, defining what unsolicited mail has implicit concent, and
writing BCPs about filters (e.g. filtering "SHOULD" be done during
the SMTP transaction and their SMTP messages "SHOULD" include clues
to help with false positives).

A BCP that says that MUAs "SHOULD NOT" generate HTML by default or at
least unless the sending user selects some formating is another good
thing that a group like this could produce, but evidently won't.

("SHOULD" and "SHOULD NOT" in the special sense of an RFC.)

---

From: "Jon Kyme" <[email-masked]>
Date: Fri, 20 Jun 2003 15:08:33 +0100

>to help with false positives).

Yep, I'm still on with the filtering at SMTP thang.  I've been
on holiday (vacation) and my wife wouldn't me do any work.

I hope to have a somewhat expanded version (formatted nicely)
for y'all next week.

If anyone missed it - the old version is here:
http://www.merseymail.com/asrg/Filter-SMTP-0-1.txt

>thing that a group like this could produce, but evidently won't.

Well, if *someone* wrote one they could bring it to the group :-)

--

---

From: Vernon Schryver <[email-masked]>
Date: Fri, 20 Jun 2003 08:17:17 -0600 (MDT)

> From: Kee Hinckley <[email-masked]>

> inline anyway).  

I disagree with all of that.  I carefully did not say anything about
writing code to look at 100,000 messages but suggested manually
examining a modest sample of messages.  For each of the issues you raise:

   - you can almost always see which formating is due to quoting some
      other message.  That formatting is gratuitous and should not be
      counted, because the receiving MUA can do it without HTML.

   - links should not be counted because MUAs automatically detect
       them in plaintext and make them "active"

   - image tags should be counted because the user presumably wanted
        the picutre

   - font changes that are not gratuitious should of course be counted

   - if you find HTML formating that you can't decide whether it's
       what the user asked for, then asssume it is not graguitious.

I predict that you'll find that other than quoting other messages and
silly noise added by rote to all of their output by lame MUAs, you'll
find essentially no uses of HTML formatting.  I claim that users almost
never ask for bold, some other font, or include (as opposed to MIME
attach) images.

>                  But the fact of the matter is that HTML email is not 

That religious statement isn't a good counter to my claim.  It also
confuses MUA presentation with sender formatting.  An MUA does *NOT*
need HTML from the sender to render quoted text in a different font,
color, or whatever.

My claim is that the vast majority of HTML mail has no formating that
the sender cares about.  If your position is ground in reality as well
as enthusiasm, it will be trivial to disprove my claim.  I don't have
access to a stream of general (non-spam) user mail, or I'd take a
sample and make the count myself.  The mail I can count (e.g. to
various mailing lists) implies that HTML formatting is even less valued
than I claim.

If HTML formating is so valuable, valued and nearly universal used,
why haven't we seen much of it in this mailing list?  Why wasn't
that text you quoted of mine in green italics or something?

---

From: Benjamin Geer <[email-masked]>
Date: Fri, 20 Jun 2003 15:41:15 +0100

On Fri, Jun 20, 2003 at 08:17:17AM -0600, Vernon Schryver wrote:
> why haven't we seen much of it in this mailing list?

Because in technical circles (and some academic ones), HTML email is
prohibited by cultural tradition -- the same tradition that says quoting in
replies should be done in the style I'm using here.  Elsewhere, another
culture reigns, in which HTML is the norm, and quoting is done by inserting
the new text above the quoted (and usually indented) message (which is not
trimmed, so that the entire thread is sent with each reply). Efforts to
impose cultural change are notoriously fraught.

Ben

---

From: Kee Hinckley <[email-masked]>
Date: Fri, 20 Jun 2003 10:42:28 -0400

At 8:17 AM -0600 6/20/03, Vernon Schryver wrote:

- image tags should be
counted because the user presumably wanted

the picutre

Well, then as a start we can probably consider the 34% of the
email which was multipart/mixed as user-requested.

But actually, even that isn't a good test.  IncrediMail
includes an ad logo with every message.  But then, from what I
can tell--most of their users signed up specifically because they
*did* want formatting.  You can have mood email, images based on
the date, background email wallpaper and lots of other fun
stuff.

That religious statement isn't a good
counter to my claim.  It also

confuses MUA presentation with sender formatting.  An MUA does
*NOT*

need HTML from the sender to render quoted text in a different
font,

color, or whatever.

Which is in fact, exactly what I said when I talked about
Eudora.

If HTML formating is so valuable, valued
and nearly universal used,

why haven't we seen much of it in this mailing list?  Why
wasn't

that text you quoted of mine in green
italics or something?

Because technical mailing lists are run by techies, and we have
actively (even on this list, if I recall correctly) berated people who
used excessive formatting.  The non-technical lists I'm on
getting formatted email all the time--even when the list software
filters it out.  I'd estimate that about 50% of the messages I
get on one non-technical email list have a message from Yahoo about
attachments having been filtered out.  People set background
wallpaper for their email!  Customizing fonts and colors is
something end-users do every day in their word processor.  They
have no idea why they shouldn't do it in their email.  Do they
*need* to?  No, of course not.  Do they want to?
Obviously.

And does a non-techie know why I just put "*"'s around
the "need" in the previous paragraph?  Maybe.
Would they do it?  No, of course not, they'd just hit the
bold button.

I will make another attempt to determine how much of that email
has user-specified formatting.  But it's very hard to quantify.
If they cut and paste a message (e.g. not forwarded) and it ends up
with the mail headers bolded--was that a user-decision (after all, the
system is only making sure the email looks just the same as it does in
the UI, and the receiving MUA won't do it) or gratuitous formatting?
Judging intention from formatting is not trivial.

And of course, it's not clear it matters here.
HTML-blocking will not work.  Microsoft isn't going  get rid
of WYSIWYG in their email programs, and commercial mailers aren't
going to drop HTML as a format for their mailings.  The whole
thing would just turn into another massive whitelisting problem.

--

Kee Hinckley

http://commons.somewhere.com/buzz/   Writings on Technology
and Society

I'm not sure which upsets me more: that people are so unwilling to
accept

responsibility for their own actions, or that they are so eager to
regulate

everyone else's.

---

From: Vernon Schryver <[email-masked]>
Date: Fri, 20 Jun 2003 08:55:21 -0600 (MDT)

> replies should be done in the style I'm using here. 

That implies that HTML should be less common here, but not as absent
as it has been.  We've had contributions from people who don't seem well
versed on such conventions, including top-posting and full-quoting.

>                                                      Elsewhere, another

Yes, but we've seen top-posting and full-quoting in this list, but
practically no HTML.  It's possible that those running Mailman for
this list have set filters against HTML, but I doubt it.

Again, I'm not saying that HTML mail is other than useful and valuable
or that the Internet doesn't carry 10,000,000's of real, intentionally
formatted HTML mail every day.  I am saying that the claims that users
care about HTML in more than a few percent of their messages are false.

You've said you have access to lots of mailing lists where HTML mail
is the norm.  Would it be too hard to look at the next 50 representative
messages?  (e.g. not a mailing list devoted to tech. writers discussing
how to do 2-column facing pages with odd/even headers and footers,
auto-numbered footnotes, and flowed text around figures).

---

From: Steven F Siirila <[email-masked]>
Date: Fri, 20 Jun 2003 10:05:09 -0500

> Netscape 7 was written by people like us, who dislike HTML email, so they've

I can tell!  Links in email header lines which used to be clickable in
Netscape 4 no longer are in Netscape 7!  To visit such a URL (for example,
to unsubscribe from a list) one has to view the raw message and cut-and-paste!

---

From: Vernon Schryver <[email-masked]>
Date: Fri, 20 Jun 2003 09:09:42 -0600 (MDT)

> <!doctype html public "-//W3C//DTD W3 HTML//EN">

>                                             image tags should be

> email which was multipart/mixed as user-requested.

That would be wrong, because we're talking about HTML formating.

> <div>But actually, even that isn't a good test.&nbsp; IncrediMail
> includes an ad logo with every message.&nbsp; But then, from what I
> *did* want formatting.&nbsp; You can have mood email, images based on
> stuff.</div>

The formating they ask for should be counted, but not the logo.

> ...
> <blockquote type="cite" cite>If HTML formating is so valuable, valued
> and nearly universal used,<br>
> why haven't we seen much of it in this mailing list?&nbsp; Why
> wasn't</blockquote>
> <blockquote type="cite" cite>that text you quoted of mine in green
> italics or something?</blockquote>

> <div>Because technical mailing lists are run by techies, and we have
> used excessive formatting.&nbsp; The non-technical lists I'm on
> filters it out.&nbsp; I'd estimate that about 50% of the messages I
> attachments having been filtered out.

We are talking about HTML formating instead of attachments, or at least
I am.

Besides your point about attachments already being filtered cuts against
your position.

>                                             People set background
> wallpaper for their email!&nbsp; Customizing fonts and colors is
> something end-users do every day in their word processor.&nbsp; They
> have no idea why they shouldn't do it in their email.&nbsp; Do they
> *need* to?&nbsp; No, of course not.&nbsp; Do they want to?&nbsp;
> Obviously.

Of course some people do.  My claim is that they are rare.  Your
claim seems to be that most or at large minority do that sort of stuff.

> ...
> And does a non-techie know why I just put &quot;*&quot;'s around
> the &quot;need&quot; in the previous paragraph?&nbsp; Maybe.&nbsp;
> Would they do it?&nbsp; No, of course not, they'd just hit the<b>
> bold</b> button.</div>

Your claim is that they do it.  Mine is that they don't.
Instead of arguing about it, how about some numbers?

> <div>I will make another attempt to determine how much of that email
> has user-specified formatting.&nbsp; But it's very hard to quantify.&nbsp;
> the UI, and the receiving MUA won't do it) or gratuitous formatting?&nbsp;
> Judging intention from formatting is not trivial.</div>

Your efforts will be appreciated.

> <div>And of course, it's not clear it matters here.&nbsp;
> HTML-blocking will not work.&nbsp; Microsoft isn't going&nbsp; get rid
> going to drop HTML as a format for their mailings.&nbsp; The whole
> thing would just turn into another massive whitelisting problem.</div>

That's a red herring.  No one has said that Microsoft should "get rid
of" their almost WYSIWYG stuff (I've worked in that area and have
opinions).  I've proposed only turning off HTML by default or
not sending HTML unless the user explicitly asks for some formatting.

> http://www.messagefire.com/&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span
> ></span>&nbsp;&nbsp; Anti-Spam Service for your POP Account<br>
> http://commons.somewhere.com/buzz/&nbsp;&nbsp; Writings on Technology
> and Society<br>
> <br>
> accept<br>
> regulate<br>
> everyone else's.</div>

As far as I can tell, all of the HTML formatting in your message
was gratuitious and not manually selected.

Note that a lot of it was incredibly bad but typical machine-generated
HTML.  For example many of the non-breaking spaces are obnoxious (but
hidden) noise HTML.  Or notice that <span></span> pair above.
But as far as I can tell, the fact that MUAs generate incredibly
bad HTML is irrelevant.

vjs

---

From: Yakov Shafranovich <[email-masked]>
Date: Fri, 20 Jun 2003 11:35:18 -0400

>versed on such conventions, including top-posting and full-quoting.

I got messages from the list that were picked up by SpamAssasin partly 
because they were HTML based, particularly some messages from Bob Wyman.

---

From: "Bob Wyman" <[email-masked]>
Date: Fri, 20 Jun 2003 12:33:04 -0400

Yakov Shafranovich wrote:
> I got messages from the list that were picked up 
> by SpamAssasin partly because they were HTML based, 
> particularly some messages from Bob Wyman.
	Well, I am not a spammer... I'll admit that I send HTML more
often than I *need* to. However, it is sometimes necessary and useful.
For instance, the list of spam related patents that I recently sent out
is much more readable as HTML than it would have been had I sent it as
plaintext since the HTML format allows me to hide the URLs that lead to
each patent document. I had, in fact, originally compiled the list as
ASCII but it was such an ugly mess that I went back and did it again in
HTML. 
	See: 
https://www1.ietf.org/mail-archive/working-groups/asrg/current/msg05356.
html

	Sometimes, HTML is very useful... And using HTML rather than
ASCII can be a service to the readers of your mail.

		bob wyman

---

From: Benjamin Geer <[email-masked]>
Date: Fri, 20 Jun 2003 17:51:09 +0100

On Fri, Jun 20, 2003 at 08:55:21AM -0600, Vernon Schryver wrote:
> messages?

OK, here are counts of non-spam messages from a list for people working on a
non-technical political campaign, for four months that had particularly high
traffic (September, October and December 2002, and January 2003).

Total messages: 144
Plain text (including quoted-printable): 94
Total HTML: 48
  'Unnecessary' HTML (just P and DIV tags): 23
  'Useful' HTML (e.g. with bold and italic, and different font sizes): 25
Entire message in base64: 2
Messages (among the above) with attachments: 4

I included quoted-printable in the plain-text category because it was needed
by several people posting messages in languages other than English.

The archives are here:

http://lists.southspace.net/pipermail/esf-uk-info

Ben

---

From: Benjamin Geer <[email-masked]>
Date: Fri, 20 Jun 2003 18:58:26 +0100

On Fri, Jun 20, 2003 at 11:36:05AM -0600, Vernon Schryver wrote:
> Not to be difficult, just to see if I agreed, I looked at October, 2002
> Of the 42 total (if I counted right), I found 6 that had real HTML
> formatting.  Three  of those were what might be called formal presentation
> documents intended for bulk distribution (e.g. seminar timetables)
> instead of person-to-person or person-to-mailing-list email.

Distribution of seminar timetables (some of which were drafts for
discussion, IIRC) was one of the purposes of that mailing list.  (It was a
list for the UK organisers of the European Social Forum, who needed to
exchange information about the seminars they were organising.)

> Was that month unusual or did I count wrong?

That month was unusual in having a lot of plain-text messages; the number of
HTML messages (of both kinds) was more consistent from month to month.

> Do you agree that documents such
> as the seminar timetables aren't strictly admissible?  I certainly 
> agree that HTML is appropriate for those documents, but they're also
> legitimate bulk mail that should be subject to white-listing.

Lists that people use for organising events often include such documents,
which then get discussed, revised, and reposted.  In any case, some
anti-spam strategies require white-listing in order to accommodate mailing
lists at all.  (Which seems OK to me.)

If you often receive event announcements (as I do), and therefore have to
read them very quickly, having the most important bits in red or in boldface
(as in a few messages on that list) can be quite helpful.

Also, on that particular list, people sometimes posted documents in plain
text, or in unadorned ('useless') HTML, which would have benefitted from
some HTML formatting, e.g. documents with headings and sub-headings.  Not
just seminar timetables, but meeting summaries, and drafts of official
declarations.  Typically people used all caps for the headings, where a
bigger font might have been more readable.

> Many of the real HTML messages included plaintext versions.
> How do you suppose their authors would have responded if it were
> suggested that HTML was a bad idea?

I guess it depends on whether a really convincing case can be made that HTML
is harmful, and whether it outweighs the perceived benefits.  In my
experience, telling people that they shouldn't use HTML for legitimate
purposes, because spammers and virus writers also use it, doesn't come
across as very logical.

> It should be noted that the Mailman archive of that list might have
> discouraged the use of HTML.  The ugliness of HTML in that vintage of
> Mailman archives is one of the reasons why I've configured Mailman to
> reject HTML submissions to the DCC mailing lists.

I know, those archives are horrible. It's one reason why I've given up on
Mailman altogether, and switched to Sympa (http://www.sympa.org).

Ben

---

From: Vernon Schryver <[email-masked]>
Date: Fri, 20 Jun 2003 12:06:36 -0600 (MDT)

> 	This is not necessarily a valid measure of utility. It is often
> the case that HTML is displayed using fonts which are different from
> those used to display "ASCII" messages. Given that those fonts are often
> variable pitch and given that variable pitch fonts are known to be (for
> most people) much more readable, one can argue that there is utility in
> sending HTML formatted messages even if that HTML doesn't include
> special rendering of text. The utility comes from sending the message in
> a format which is likely to be easier to read as well as to be more
> readily understood by the reader.

That's wrong.  If you as a mail receiver like variable pitch fonts,
then you can have them with common MUAs with plaintext mail simply by
configuring your MUA.  If you don't like variable pitch fonts, then
you also don't need to see them with HTML messages that don't specify
a font.

Do you think much HTML mail specifies a font by other than <BIG>
or a numeric size?   My impression is otherwise.

---

From: Vernon Schryver <[email-masked]>
Date: Fri, 20 Jun 2003 11:36:05 -0600 (MDT)

> http://lists.southspace.net/pipermail/esf-uk-info

Thanks!  

So by your count, 25/144 or 17% involved real HTML?  
That's more than I predicted, but less than it seemed others suggested.

Not to be difficult, just to see if I agreed, I looked at October, 2002
Of the 42 total (if I counted right), I found 6 that had real HTML
formatting.  Three  of those were what might be called formal presentation
documents intended for bulk distribution (e.g. seminar timetables)
instead of person-to-person or person-to-mailing-list email.  Was that
month unusual or did I count wrong?  3/39 or 8% is above my prediction
but significantly below 17%.  Do you agree that documents such
as the seminar timetables aren't strictly admissible?  I certainly 
agree that HTML is appropriate for those documents, but they're also
legitimate bulk mail that should be subject to white-listing.

---

From: "Bob Wyman" <[email-masked]>
Date: Fri, 20 Jun 2003 13:47:55 -0400

bob wyman

---

From: Kee Hinckley <[email-masked]>
Date: Fri, 20 Jun 2003 17:41:06 -0400

>was gratuitious and not manually selected.

In which case you would be wrong.  It was triggered by my making one 
single word bold.

>But as far as I can tell, the fact that MUAs generate incredibly

It certainly could have done a better job, but since it was Eudora, 
and the the Eudora developers pretty much despise HTML email, I 
suspect they didn't spend a lot of time trying to make it efficient. 
After all, even with all that mess--it's still better than what 
Outlook generates.

However.  The real point of my response.

You asked why, if HTML was so common, we didn't see more of it on 
this list.  My posting (as I hoped) quickly generated a perfect 
example of why technical mailing lists don't have a lot of HTML email.

At 2:36 PM -0600 6/20/03, xxx wrote:
>If whatever you said is important, I'm sure you'll send it again without
>the extraneous unreadble code.  ;-)
>
>Please don't send HTML to mailing lists.  Like any MIME attachment, HTML
>should be sent only when you know the recipients can (and will) decode it
>and make user of it.  Chances are you don't know all the recipients of a
>mailing list will be able to handle it.

Thank you to the person (who shall remain anonymous) who sent me that 
off list.  You couldn't have said it better if I'd planted you there 
:-).

---

From: Vernon Schryver <[email-masked]>
Date: Sat, 21 Jun 2003 11:46:29 -0600 (MDT)

> I would guess that's not unusual for a non-technhical mailing list. 
> I suspect person-to-person (letters from your Mom, that sort of 
> things) will have a higher HTML ratio, but we'll see.

We're talking about real HTML, not HTML that is present only
because Netscape and Outlook generate HTML by default.  Perhaps your
Mom is different from most people who use computers only casually.
Most people are less likely to use text formatting than the apparently
fairly computer savvy contributors to that mailing list.

Someone mentioned something about an "apple-B" key apparently in
connection to making a word bold.  That is meaningless to the
majority of users who have PCs instead of MACs.  For several reasons,
I wouldn't be surprised if MAC users are more likely than average
to use real HTML formatting.

I'm irked at the demands to squelch this topic.  This topic has at
least as much hope of having some useful result as the zillions of
words spent on "CR" and "CRI."  It makes no sense to talk about
filtering HTML by default for the immediately foreseeable future, but
BCPs on aspects of spam, virus, worms, or privacy could quite properly
and valuably urge that HTML be used only when really required or
explicitly requested.  The current practice of bloating mail by 10X
encoding with a text/plain version and a visibly identical text/html
version is nasty, bad, and incompetent.

---

From: Kee Hinckley <[email-masked]>
Date: Sat, 21 Jun 2003 11:33:17 -0400

---

From: Kee Hinckley <[email-masked]>
Date: Sat, 21 Jun 2003 13:35:50 -0400

>configuring your MUA.  If you don't like variable pitch fonts, then

Which works fine until you get email from someone who assumed you 
were using fixed pitch.  I've tried it, it's not worth the hassle. 
Lots of plain/text messages don't assume fixed pitch, but when they 
do, they *really* do.

>or a numeric size?   My impression is otherwise.

Absolutely, all the time.  In fact, in the code I just wrote to look 
for hand-generated HTML, I specifically count font tags as hand 
generated only if the face or the size changes from what was used in 
the previous font tag.  If I just looked to a face tag, just about 
everything from Outlook would get marked as hand-generated.  Even my 
pre-teen daughters send multi-font email.  Don't look at me.  *I* 
didn't tell them how.  In fact, in one of my older daughter's first 
messages to a friend, she figured out how to cut and paste the smiley 
graphics from iChat into the mail program, and mailed off a message 
containing 200 itsy bitsy TIF images of different smiley faces. 
(Bummer drag--first message to someone, and it was HTML.)
-- 
Kee Hinckley
http://www.messagefire.com/          Anti-Spam Service for your POP Account
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: mathew <[email-masked]>
Date: Sun, 22 Jun 2003 11:00:24 -0400

On Saturday, Jun 21, 2003, at 13:46 US/Eastern, Vernon Schryver wrote:
> The current practice of bloating mail by 10X encoding with a 
> text/plain version and a visibly identical text/html version is nasty, 
> bad, and incompetent.

Yes, but it's also totally outside the purview of the ASRG, unless you 
think it is in some way an important enabler of spam.

I wholeheartedly agree that unnecessary use of HTML in e-mail is 
stupid. I completely concur that HTML e-mail is generally a waste of 
bandwidth. However, as far as I can see these arguments over how 
technically awful HTML e-mail is are irrelevant to this list, because 
it's not going away and even if it did spam would just go back to being 
plain text.

(In fact, all the spam that's made it through my filters this week has 
been plain text.)

mathew

---

From: Vernon Schryver <[email-masked]>
Date: Sun, 22 Jun 2003 11:17:24 -0600 (MDT)

> From: mathew <[email-masked]>

> (In fact, all the spam that's made it through my filters this week has 

How much HTML spam did your filters reject before it got to your mailbox?

If HTML vs. spam spam is off-topic here, then so are all other anti-spam
mechanisms that allow mail among strangers.  They are all much less
then perfect and so by your criteria must be off topic:

  - RMX/etc must be off topic because unsolicited bulk mail can and
     often is sent with completely valid sender information.  Spammers
     need only switch to sending from "bulletproof hosting bureaus"
     to be invulnerable to any sort of reverse DNS mechanism.

  - any form of CR/CRI that involves automated or potentially automated
      (e.g. text only) challenging must be off topic because spammers
      need only have their computers answer the challenges.  Since there
      is no immediate prospect of a computer that can pass a Turing
      test, it is impossible for a coputer to pose a challenge that
      cannot be passed by another computer given sufficient processing,
      including pictures and sounds.  Thus even challenges that are
      intended to be answered only by humans can in principle be
      "scripted."

  - any sort of authentication as a spam defense that still allows
      receiving mail from long lost friends is off-topic, because spammers
      need only buy new, authenticated identities from Verisign or
      other identity vendors as needed.

  - any sort of sender-pays system is off topic because spammers need only
      pay to send it.  Junk postal mail advertisers are willing to spend
      up to $0.50 (US) per target, and that's far more than any proposed
      sender pays system.

  - laws against spam are hopeless because many spammers now
     enthusiastically break existing laws.
      
And so on for absolutely every scheme including blacklisting, graylisting,
Bayesian filtering, keyword filtering, SpamAssassin, and my personal
favorite, the DCC.

Of course it is silly to talk about ISPs or any large organizations
filtering HTML by default.  That does not imply that it would not 
be good and profitable to write a BCP saying (among many other things):

    Filtering HTML mail from strangers is an effect spam defense and
    should be considered a 1%-10% false positive rate can be tolerated.
    Because some individuals and organizations do filter HTML mail
    from strangers, you SHOULD NOT send HTML or mail involving
    unnecessary MIME attachments to strangers.  Like any and all spam
    defenses that allow mail from strangers, filtering HTML mail is
    only partially effective.

As for the talk about MIME attachements--yes, of course digital
signatures are nice, but they're almost always a waste of bandwidth
on mail from strangers.  A 4 KByte signature block in a public mailing
list or in a private message from a stranger is evidence only of the
sender's mistake, ignorance, or egotism.  MIME attachments of supposed
signatures on mail from strangers prove nothing more than the MIME
attachements of supposed PGP signatures on spam.   (I hope (or wish)
anyone who presumes to talk about spam here has enough experence with
spam to have seen plenty of spam with MIME PGP attachements.)

For base64 and quoted-printable--again, how do you expect people in
China to talk to each other if don't allow them to encode their language
in what is still fundamentally a 7-bit ASCII, ANSI x3.-4, transport,
where ASCII stands for "American standard for character information
interchange" and does not even allow the English to talk about their
money?  Besides, only toy or snakeoil spam filters are the least
bothered by Base64 or quoted-printable and neither costs significant
bandwidth.

---

From: mathew <[email-masked]>
Date: Sun, 22 Jun 2003 15:34:36 -0400

On Sunday, Jun 22, 2003, at 13:17 US/Eastern, Vernon Schryver wrote:
> mathew <[email-masked]>:

I don't know how much was rejected because it was HTML; I use 
SpamAssassin, and the HTML-ness is just one of many factors it weighs.

> then perfect and so by your criteria must be off topic:

I did not say that anti-spam mechanisms which are imperfect are 
off-topic. I said that HTML e-mail is not necessary for spam to exist, 
and that rich text e-mail is not going to go away.

Hence all your paragraphs of things which "must be off topic" are 
irrelevant straw-man examples.

> filtering HTML by default.

I'm glad you agree with me on that.

>   That does not imply that it would not be good and profitable to 
> write a BCP saying (among many other things):
>

I'm pretty sure nobody on the list is going to disagree with that 
either.

> money?

Yet another good reason why HTML and base64 aren't going away.

mathew

---

From: Bruce Stephens <[email-masked]>
Date: Sun, 22 Jun 2003 23:26:11 +0100

mathew <[email-masked]> writes:

[...]

> Yet another good reason why HTML and base64 aren't going away.

Indeed.  If users are unhappy with text/plain (and want bold, italics,
etc., reflowing paragraphs and so on), then we've already got a
non-HTML solution: text/enriched, rfc1896.  

But let's be realistic: we're stuck with HTML.  The best we can hope
for is to get software producers to default javascript to off, and
similarly for retrieving remote URIs (so we can stop webbugs from
working).

---

From: Vernon Schryver <[email-masked]>
Date: Sun, 22 Jun 2003 17:47:40 -0600 (MDT)

> From: Bruce Stephens <[email-masked]>

> non-HTML solution: text/enriched, rfc1896.  

What common MUA's can generate or even display RFC 1896 rich text?

> similarly for retrieving remote URIs (so we can stop webbugs from

That is realistic, but we could also hope to get software producers
to turn off HTML by default, or at least until the user asks for
some formatting.   

That hope, very faint as it is, is less unrealistic than hoping that
the same software producers will include support for CR/CRIs, RMX, or
about anything else that they are not already supporting.  Turning
off HTML until the user asks for formatting would not cause user
consternation, unlike including spam rejection facilities.

---

From: mathew <[email-masked]>
Date: Tue, 24 Jun 2003 21:57:56 -0400

Eudora.

> to turn off HTML by default, or at least until the user asks for

Like Apple Mail does, for example.

mathew