Subject: [Asrg] Collected comments on spamming, permissions lists, et al

From: [email-masked]
Date: Sun, 22 Jun 2003 13:55:11 -0500

>>> (Could you adjust your line length to less than 78?  My ancient MUA
> doesn't rewrap lines, and I think it shouldn't.  I control my own line 
> lengths by piping paragraphs through `fmt` as I edit them with `vi`.)

>> Could, but in my experience the artifacts of doing that tends to
> be worse rather than better, since one ends up with chewed-up
> interlaced paragraphs that are even more frustrating.  Sorry.

> What's that about interlaced paragraphs on text with proper lengths?

Presently my mailer is set when composing messages to set a line break at column 
80.

If I set it to less than that, then quoted lines which are 82 columns wide 
(say)
end up looking like this.  This can make for very ugly paragraphs, unless
I end
up going through the paragraphs and reformatting them manually (which I'm
not
always willing to take the time to do.

This is especially true on paragraphs which have been quoted several times in an 
exchange and grow (due to the quoting indicators) at each level of quoting, and 
then broken several times.  These paragraphs get really ugly.

This is one reason why I usually MANUALLY insert a quote level indicator only 
just on the first line of a quoted paragraph... to prevent quote level line size 
creep on every line of quoted paragraphs.

But that's just my own convention which works for me.  Obviously you have the 
right to choose differently.

> Typing like an old fashioned typiest and hitting "Return" before the
right margin yields text that is far more intelligible.  

Feel free to do that if you prefer.  To me, not having to hit ENTER on every 
line is part of the advantages of having a computer instead of a typewriter. :-)

> It seems
you've switched to no hard line breaks except at the ends of paragraphs.

I haven't changed anything in my mail software configuration in quite a long 
time.  I've pretty much given up trying to figure out all the vagaries of 
processinng that E-mail can sometimes go through enroute.

> That's better, or less bad, than your previous 79-83 character lines.

Again, my composing is set to insert a line break at 80 columns.

> I can send a 300 character paragraph through `fmt`, quickly add '> '
to the beginnings, and produce something that's legible in the archives.
79-83 character lines are a royal pain to quote.

I don't use "fmt".  That's okay, you don't use MY preferred mail software, 
either.  :-)

>>> ...until ISPs can tell users who complain "run the Netscape or
> Microsoft auto-update mechanism", they will not and cannot do such
> filtering by default.

>> I'm not sure I understand your comment.  The reason they would
> want to change THEIR mail software would be so it would NOT generate
> HTML-burdened E-mail by default.  To truncate the undesirable/untrusted
> stuff earlier on the transmission/delivery path (BEFORE wasting as

> Fine, but no non-trivial ISP that wants to keep its users, and for that
matter, no "IT" department that wants to keep its empire could start
filtering incoming HTML mail by default this month or even this year.

Actually, for an ISP, they'd probably phase it in.

There are a whole variety of imaginable ways to handle that.

For example, one approach might be that all 'would be filtered' E-mail would 
deliver to the recipient along with a note (either prepended/appended or as a 
separate E-mail) that warned about the upcoming change and the affected E-mail, 
and offered an easy way to deal with it.  Perhaps they'd be offered the choices 
of "allow such mail in the future arriving from this sender", "mail the sender 
an informative form letter asking them to send to me in plain ASCII format 
instead", "bounce such mails from this sender in the future" or just "t-can such 
mails from this sender in the future".  The warning should point out that a 
default blocking of such mails from infrequent/unknown/untrusted senders 
(including virtually all spammers and worms/trojans) will take place after 
(whatever future date) unless or until the recipient wishes to 'opt out' of the 
protection feature.

How those choices might be implemented and relayed to the ISP could vary 
depending upon the particular needs of the ISP or company and the type of 
procedures and systems they use.

> Local users would revolt because a large fraction of their incoming mail
would be bouncing.  Until mail that is legitimate except for gratuitous
HTML encryption is largely gone, no large outfit can filter HTML by default.

I suspect that the larger outfits will come onboard slower, perhaps kicking and 
screaming (despite the fact that THEY will have the most to gain by doing so).

I'd guess that smaller ISPs will initially see this as a major opportunity to 
differentiate their services from those of "larger, less responsive" ISPs.

> The most that any large outfit can offer is to filter HTML mail for
users who ask for such filtering.  

I don't agree on that point, although I do suspect that there will be a 
transition period to help the user to build up their "permissions list" for 
those who frequently send mail to that recipient.  I doubt that the transition 
period will need to be more than two or three months, since frequent senders 
will probably all be approved by the end of that time.

One might imagine a secondary stage of the transition, during which time the ISP 
might reroute offending messages to a different mail queue (to hold them for a 
few days, or even weeks perhaps) and send the recipient (say, daily, hourly, 
whatever,,, maybe once per E-mail pickup run?) a list or digest (maybe the 
headers and the first three/ten lines of the body?) or something of the details 
of the mails which would have been bounced or blocked, and giving the user a 
simple way for each on the list to:

  1)  deliver that message to them on a one-time basis
  2)  deliver that and similar messages in the future (i.e. give permission)
...(again, or other options...)

This would spare them delivery of the unwanted messages as a transitional stage, 
while still allowing them to see what would have blocked.  The pending queue 
could hold messages for a while, or eventually discard them or bounce them if 
not eventually picked up or otherwise dealt with.

Perhaps the user would be given the choice of at what point they're content to 
skip the need for this daily "messages held pending list" entirely and progress 
to full automatic bounce-or-discard processing.

> There are many ways to do that today.  I think one is to use a Milter version 
of SpamAssassin, although I'm not sure if all of those support per-user control 
files.  

The key problem with that, as a primary approach, is that filtering 
only-at-the-end-user means that essentially all of the cost of the handling of 
the spam has already been incurred.  

> I've seen plenty of headers on mail saying that base64 encoding has
beem removed.  I've also seen plenty of spews with some samples
encoded and some not.

> But regardless, what's your point about Base64?  It does nothing to
help spammers or hurt filters.  

It *does* complicate content filtering.  It's another stage that must be 
performed before the content can be examined for "suspicious" content.

> The first is why relatively little spam is current base64 encoded.  

Hopefully that will continue to be true.  :-)

> Base64 encoding wastes 30% of bandwidth and disk space, but that's not a 
reasonable concern given the small fraction of total traffic that is all email. 
 
If we don't have to worry about E-mail bulk, then we're mostly wasting our time 
here.

The lawsuits that HAVE been placed against spammers by large ISPs usually are 
predicated precisely on the fact that the spam wastes ISP resources and costs 
them a significant amount of money.

> HTML as it is commonly processed by receiving MUAs has many evils, but base64 
is merely a trivial waste of bandwidth.

Trivial?  Maybe, maybe not.  I'd be willing to forego (for now, at least) the 
ability to by default drop out base64-encoded message bodies from the default 
permission list filtering, as long as we'd get the HTML-burdened mail filtering 
and attachments filtering (perhaps with further subselection by attachment 
type... lots of correspondents and mailing list who might send me a PDF/GIF/JPG 
are folks who I would *never* want to accept an EXE/SCR/PIF/VBS/etc file from!).

[snip]

>> ...we do have a well-established and longstanding convention that plain
>ASCII text is the _most_ universal form of E-mail that EVERYBODY can read on
>virtually ANY kind of device.  It makes sense, until a sender knows that 
>other
>types are also acceptable, to send in most widely readable and acceptable
>format.

> ...As for a "longstanding convention", due to the many MUAs sending HTML 
email, that convention has been changed and now HTML email is acceptable.

It doesn't change just because some thoughtless and inconsiderate (or merely 
ignorant) senders choose to send mail which doesn't satisfy it.  

Not everybody reads mail using Web browsers, and even more people don't read 
E-mail online (this is especially true outside the USA where many users still 
pay a per-minute-local-call charge to connect with their ISP, and thus prefer to 
read and respond to their E-mail offline).  Some people are even reading and 
responding to E-mail using their notebook computers at 33,000 feet (and are 
DEFINITELY not doing THAT online, at least at present).  For these people, it is 
highly annoying to get an E-mail that includes text-image fetches from Web 
servers and other forms of Web bugs in it.

But again, if in fact this is really an "acceptable" format for everybody, and 
they don't see value in using it to help throttle spammers and their abuses, 
then they'll just all set the filters to 'wide open' and things will be no worse 
than at present.  

[snip]

> First of all, making sure email is not untraceble allows 
for LEA to catch the spammers. 

Unforunately, there ARE legitimate cases ('whistleblower' E-mails or e-mails 
from repressive regimes) where untraceability can be of primary importance.

> This would involve either changing SMTP, 
implementing C/R, or some other system that would allow for traceability. 

All of those approaches seem to require a consensus, a willingness to change 
globally.  I don't see that happening, especially if some strongly 
civil-rights-to-privacy-oriented ISPs refuse to go along.

I personally don't think that we MUST enforce "untraceable" E-mail, as long as 
we offer recipients the option to say that they simply don't want to receive 
that type (or other types) of mail from such shady senders.  This gets back 
again to the "permissions list" concept I've proposed.

> Domain names being owned by spammers is a problem too. Solutions must be 
made to deal with that as well. Foreign ISPs, allowing for spam are also a 
problem. 

Frankly, I don't think you'll EVER solve that one.  Better to just simply give 
the recipient the control to block the risky/bulky/obscured stuff they don't 
want, from any people they don't recognize.

Of course, another option might be offered in the permissions list approach, and 
that's kind of one of the functions of my own incoming mail filtering system... 
and that's to say "Allow incoming E-mail that's HTML-burdened, but strip out all 
the HTML and such that looks unnecessary or suspicious."  This way, even if Aunt 
Bertha hasn't been added to my list of trusted recipients, her mail wouldn't be 
blocked just because she unthinkingly tried to boldface one word in it... the 
offending <b> and </b> tags would just be silently removed in transit.

Again, this isn't a general proposal, but it IS an option that might be offered, 
and might be an alternative to bounce/t-can HTML-burdened incoming messages.

One could even develop a more complicated and sophisticated concept which would 
further differentiate HTML based on what tags were used... where the stuff Aunt 
Bertha or other fairly clueless users might be inclined to use (fonts maybe, 
italic, boldface, underline) would be tolerated but stuff that Aunt Bertha would 
probably never use (scripting, tables, frames, cascading style sheets, whatever) 
would trigger the anti-spam defenses.  Again, there are many possible variants 
on these ideas and much of the control ought to be left to the choice of the 
recipient.

> And as you have mentioned many times before computers infected 
with viruses and other similar junk are a problem as well, although I do 
not see any possible solutions for that as well, not even any avenues of 
research.

The only viable defense for that IMHO is to prevent them from getting infected 
in the first place, and blocking attachments from people who aren't normally to 
be expected to send them (and with possible subapproval by types) would go a 
long way towards solving that problem, too.

[snip]

> If a large number of systems on the internet start depending on whitelisting, 
then spammers will start seeking out ways of sending you email from 
addresses that you have probably whitelisted. 

That's certainly true.  That's one reason why you want to minimize the number of 
people who you give those special permissions to (and why well-known large 
senders will want to not require those special permissions, since it will open 
their identities to such kinds of impersonation frauds).  That's a good reason 
for newsletter senders (for instance) to send plain ASCII text versions of their 
newsletters... apart from the sheer bulk issues.

[snip]

> It makes no sense to talk about filtering HTML by default for the immediately 
foreseeable future, 

I certainly don't agree with that.  One approach that might work during a 
primary transition period is discussed above.

> ...but BCPs on aspects of spam, virus, worms, or privacy could quite properly
and valuably urge that HTML be used only when really required or
explicitly requested.  The current practice of bloating mail by 10X
encoding with a text/plain version and a visibly identical text/html
version is nasty, bad, and incompetent.

On that point, I *strongly* agree.  [standing ovation!]  Once people realize the 
costs and hazards involved, I think we can ratchet back a lot of that... AT 
LEAST by spammers.

[snip]

> Unfortunately, restricting email content type seems unlikely to have
any long-term impact on spam, as the spammers will simply adapt their
behavior to whatever content type is most likely to get their message
across.

That claim is self-contradictory.  If spammers change their behavior to send out 
plain ASCII text spam messages, since that is the only type certain to get 
through from previously unknown senders, then we *have* had a long-term impact 
on spam... if _nothing_ else, by reducing its volume by 70% or more.

Gordon Peterson                  http://personal.terabites.com/
1977-2002  Twenty-fifth anniversary year of Local Area Networking!
Support the Anti-SPAM Amendment!  Join at http://www.cauce.org/
12/19/98: Partisan Republicans scornfully ignore the voters they "represent".
12/09/00: the date the Republican Party took down democracy in America.

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: mathew <[email-masked]>
Date: Sun, 22 Jun 2003 15:27:17 -0400

> creep on every line of quoted paragraphs.

Learn to use a text editor. You're on a holy quest to get people to 
give up HTML e-mail, yet you yourself can't work out how to edit and 
quote text in a plain text e-mail? Is this some sort of comedy routine 
you're performing for our amusement?

> right to choose differently.

I choose not to read any more badly-formatted crap you send. Shape up 
or get killfiled.

mathew