Subject: Re: [Asrg] Re: TitanKey and "white lies"... (Faking SMTP hard errors "improves" C/R utility?)

From: Scott Nelson <[email-masked]>
Date: Mon, 02 Jun 2003 10:44:49 -0700

At 07:21 AM 6/2/03 -0600, Vernon Schryver wrote:
>> From: Scott Nelson <[email-masked]>
>
>> ...
>> Never use absolutes, they will always get you in to trouble.
>
>Many absolutes, including that one, can get you into trouble.
>
>> Suppose that all challenges included the Message-Id: of the 
>> quarantined message in it's In-Reply-To: and/or References: header.
>> (like all DSNs should - grrr)
>>
>> Then as long as mail clients can recognize message-ids they've created,
>> it's a simple matter to accept any you've generated, and discard any
>> that you haven't.  (Handy for those forged return address bounces too.)
>
>So few users know what a message-ID is, and so few of those who do
>know have the least clue about the message-IDs of their last dozen
>sent mail messages that it is safe to make the absolute statement that
>no users could ever distinguish valid from bogus spammer challenges
>with message-IDs.
>

/I/ could, but then, I've studied the problem intently, and
my messages-ids are quite a bit different from the "standard" ones.

More importantly, my mail client can do it.
Anyone mail client that used a similar method could do it too.

>
>> If you're worried that spammers will start forging plausible ids 
>> for their In-Reply-To: headers, or that's too hard for your computer
>> to remember a few thousand IDs, here's a simple way to create them.
>> id = "UTC.N@" . md5sum(UTC, N, domain, secret);
>> ...
>
>How many people are going to manually check the last few days of mail
>they've sent looking for the message-ID of a new challenge?  Obviously
>no one, including practically all C/R advocates will bother.  If any
>users were willing and able to do that, there would not be a continuous
>flood of questions from people asking how their computers managed to
>send spam when they weren't looking when they get a forged bounce.
>
>Even if you modify everyone's MUA to record the Message-IDs of outgoing
>mail (no, not all MUAs do that, not even when configured to record
>outgoing mail), the absolute statement that no one would manually
>check Message-IDs is still more than "5-nines" true.  You would have
>to modify MUAs to automagically check "sent folders."
>

If your mail client creates them in a way that can be checked,
then it doesn't have to remember anything except the rule for
generating them.  I described a way to do it.  I've been testing
a similar method for a while now, without any problems.  

If anyone is working on a something that generates message-ids, 
and is having trouble understanding or implementing the idea,
I'd be glad to help them.

>This sub-thread started with the observation that many of us can't
>remember to whom we sent mail, and you want us to remember Message-IDs?
>If manual (or automatic) checking of Message-IDs could work, then so
>would manual (or automatic) checking of "sent folders" for recipients
>(except for .forward files, aliases, etc.).
>

I /want/ MUAs to be able to identify their own IDs, yes.
I pointed out a way that could happen that involves changing MUAs.
I don't expect that change to occur overnight, 
or achieve universal acceptance.

Examine the pieces of what I'm talking about, and consider
the cost vs. benefits.

DSNs (including challenges) need to include an "in-reply-to:" header.
    For most this is a change, but not a difficult change to implement - 
it doesn't require esoteric knowledge, or involve a third party.
There is a tiny benefit, in that the tiny percentage of people who
actually pay attention to message-ids are less confused.  As more
people do it, the benefit increases.  The message-ids are compatible
with the legacy system.  The major cost is the cost associated 
with any change.   I wouldn't expect anyone to roll out a new
version just for this, but I hope that many will add it in and
roll it out with their next revision.

MUAs need to generate messages-ids that they can recognize.
    For most this is a change, but not a difficult change to implement - 
it doesn't require esoteric knowledge, or involve a third party.
There is a tiny benefit, in that the tiny percentage of people who
actually include the id in DSNs can be dealt with better.  As more
people do it, the benefit increases.  An additional benefit is that
stupid address scrapers wouldn't mistake a message-id like 
<20030602T151721@07107eb572829bb7302d6f05ca873309> for an address, 
unlike <[email-masked]>.  (This is not as big a problem 
as it used to be, scrapers are getting smarter.)
The major cost is the cost associated with any change.   
I wouldn't expect anyone to roll out a new version just for this, 
but I hope that many will add it in and roll it out 
with their next revision.

Identifying your own message-ids:
    For most this is a change, but not a difficult change to implement - 
it doesn't require esoteric knowledge, or involve a third party.
There is a tiny benefit, in that the tiny percentage of people who
actually include the id in DSNs can be dealt with better.  As more
people do it, the benefit increases.  An additional benefit is that
DSNs which do include a "in-reply-to:" header that can be determined
not to have been generated by the client can be discarded.
I wouldn't expect anyone to roll this out until after a few 
percent have actually started including message-ids in their 
DSNs/challenges.  If the Earthlink challenge system does it, 
or qmail, or sendmail, then that might be enough right there.
The costs though small, are much larger then the previous two,
since this involves a change to the user interface (identifying
good/bad DSNs isn't much use unless you can communicate that to
the end user)  The benefits are larger though, since this has
the potential to actually block some spam.

All these things are changes, and changes take time.
But the costs are small, and the potential benefits large.
Most spam isn't going to look like a challenge until challenges 
are wide spread.  Either few have implemented the changes, 
in which case counterfeit challenges have little value,
or many have implemented them, in which case counterfeit 
challenges very hard to manufacture.

>
>> IMO the killer problem with C/R is the "automated notice of 
>> something important" message.  Frequently, those don't even have 
>> a valid return address, much less a human that will click 
>> on your web site.
>
>That's also a killer.
>
>C/R systems depend on and will in practice devolve into whitelisting
>systems.  It would be good to finish the C/R protocols, IDs, or whatever,
>and move on to the whitelisting mechanisms that C/R systems require.
>That's where most of the utility will lie.  You'll need protocols or
>some sort of mechanical support to exchange self-signed certs, PGP keys,
>or whatever to foil spammer attacks on the whitelisting system.
>

Even with a crypto-secure mail system, 
I can't see any way to prevent someone on whom you're dependant 
from abusing that and spamming through the whitelisted channel.  
My best answer so far is "If spam only came from the companies 
you dealt with, that's better than the current situation.".  

IMO the friends list is the weakest link, and it's also the most 
difficult to fix.  If forgery was the /only/ way to send spam,
then almost all spam would be forged.  I can see ways to prevent
forgery, but they all involve esoteric knowledge, trusted third 
parties, and require large scale acceptance before any real
benefit is realized.  Most people would balk at rejecting
a sender that isn't certifiable, even if 90% of the senders are.
Even Microsoft would have trouble achieving 90% penetration.

OTH, it might be possible to at least get "automated notice of 
something important" messages to be digitally signed, sent from 
trusted IPs or whatever whitelist mechanism becomes a standard 
(assuming of course that there ever is a standard).

Scott Nelson <[email-masked]>

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Vernon Schryver <[email-masked]>
Date: Mon, 2 Jun 2003 14:21:01 -0600 (MDT)

> with their next revision.

What is the reasonable basis for your hope?  Certain large vendor(s)
not only can't even get things like "out-of-office" messages right but
insist that their broken wrong why is the right and ony way.  Why
would they generate your flavor of Message-ID, assuming it is all you
say it is and more?  Then there are the people who use qmail despite
the fact that when qmail acts as an MUA (e.g. generates a DSN or when
fronting for a mailing list), it often does not generate any Message-IDs,
and these are exactly the early-adopters you would hope to use your
new flavor.  As you say, the benefits for generating your message-IDs
start at zilch and with noticable costs (e.g. release cycle), and
slowly improve over the years.  I'm not saying it can't happen, but
asking for a better reason than "it would be nice" and "maybe someday
it will help somewhat."

(I know that qmail often does not generate Message-IDs because detecting
their absense is one of my better spam filters.  Qmail systems are
the only significant false positives for my mail and the most significant
false positives for the 1,000,000's of mailboxes protected by the DCC.)

> But the costs are small, and the potential benefits large.

I'm made bazillions of low cost eventually high profit suggestions in
zillions of areas over the last 35 years.  If even 10% of them had
been adopted, we would be in heaven...or something.  I've long since
realized there are two kinds of changes, those that I can make personally
or that can be made by people I know and can convince, and very long
shots that are almost always wastes of time to even talk about except
while "shooting the bull."  Convincing people you don't know at
addresses you can't even describe to do something like change their
message-ID generating code for a possible reward in several years is
one of the second kind.  It could happen, but we better have a fall
back defense against spam.

> are wide spread. 

Yes, and C/R systems are unlikely to ever be widespread.

I wish the C/R advocates would get on with specifying their protocol
or whatever in an I-D and then RFC so that the world can move past
that chimera.

> you dealt with, that's better than the current situation.".  

On the contrary, junk mail from your correspondents "scales."
Practically no one's correspondents increase with the size of the
Internet (at least not this century).  The melt-down problem of spam
would not apply if you only received junk mail from people and outfits
you know.  Besides, when one of your whitelisted correspondents goes
over to the dark side, it takes only a moment to fix the problem
forever.

> Even Microsoft would have trouble achieving 90% penetration.

I agree with that, but others seem to have other views.

> (assuming of course that there ever is a standard).

You keep saying things related to that notion of "important messages"
that I don't understand.  All spam are important messages according to
their senders--no sarcasm intended.

Vernon Schryver    [email-masked]
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg