Subject: RE: [Asrg] TitanKey and "white lies"... (Faking SMTP hard errors "improves" C/R utility?)

From: Yakov Shafranovich <[email-masked]>
Date: Sun, 01 Jun 2003 01:27:12 -0400

At 09:24 PM 5/31/2003 -0600, Vernon Schryver wrote:

> > From: Yakov Shafranovich <[email-masked]>
>
> > ...
> > >a) I am a sender with a CRI system and send a message to a recipient 
> with a
> > >CRI system.  My CRI system remembers I sent a message to a particular
> > >recipient.
> > >b) The recipient receives my message and holds it.  The recipient 
> sends me a
> > >challange message with CRI headers.
> >
> > If we were using some form of SMTP-model, then the recipient would not
> > actually accept the message. Instead his system would issue some form of
> > SMTP error code back, something like "450 Need to verify sender". This 
> will
> > force the sender's system to hold on to the message until the C/R process
> > runs its gamut.
> > ...
>
>SMTP clients that merely follow RFC 2821 without knowledge of your
>CRI system will retry the message on their normal schedule.
>See section 4.5.4.1 of RFC 2821.
>
>You are surely not assuming that all SMTP clients in the Internet will
>be replaced before you turn on your CRI system.  So you must be assuming
>that SMTP servers can recognize retransmissions of the same message
>or same sender and not challenge a second time.  Have you considered
>the practical difficulties of that?

I was thinking of defining such SMTP protocol not for general use, but 
rather as an alternative for cases where both the sender and receiver have 
C/R systems and want to shift the storage of messages to the sender. This 
would be accomplished by defining an ESMTP extension that is following 
these guidelines, NOT via general SMTP traffic. All existing clients and 
SMTP servers, would use the general CRI protocol as defined by Eric. Any 
C/R systems that are capable of performing C/R via ESMTP, would agree to 
use it with an option to fallback to the general CRI protocol. As for the 
re-transmissions, the unique C/R token can only be used once and would stop 
the message from being re-transmitted more than once.

>By the way, how do you handle MX secondaries?  What if the SMTP client
>happens to choose different MX servers for its attempts to send the
>message?  How do you avoid sending a challenge from each MX server,
>in practice and not merely theory?

Any C/R system at least on the receiver's end would have to keep state in 
order for C/R to work. However, my understanding that in a general C/R 
system the MX servers are not required to keep that state information but 
rather that can be offloaded to the the back-end system, with the MX 
servers just accepting the transmissions. In practice I do not know what 
existing C/R systems do for state information - whether its kept on MX 
level or back end (perhaps some of the C/R people on the list can provide 
information on that).

In this system that I proposed, such state information would have to be 
shared among all the MX servers and the obvious problem with that would be 
that such sharing requirement would be hard to implement in large systems. 
I am not advocating this specific approach, just using it as an example of 
how the entire C/R handshaking process can be done via some direct protocol 
instead of having the receiver's system accept and parse the message before 
sending a challenge. Some form of a direct C/R-to-C/R protocol would help 
interopable C/R systems save on storage costs and perhaps it should be 
something we add to the draft CRI specification.

Yakov  

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: "Eric Dean" <[email-masked]>
Date: Sun, 1 Jun 2003 10:46:33 -0400

> SMTP error code back, something like "450 Need to verify sender".
> This will

Yes, this would be about as efficient as RMX.  I'm guessing that the CRI
systems would have to first negotiate their capabilities.  In other words,
if I send a 450, I don't know if the remote system is churning through it's
database to look for a message sent to me or not.  If someone says they are
CRI capable, then I should not accept a message if they do not supply a
Response String.  If they aren't CRI capable, then I should accept the
message and do whatever I do with mail.

Another area of concern is that an SMTP method does not allow for client
software to perform the CRI process.

---

From: "Peter Kay" <[email-masked]>
Date: Sun, 1 Jun 2003 21:44:15 -1000

There have been several requests for this, so, no problem. We've got
several accounts that get loads of spam. We'll post out some CSVs of the
bounce data and you can all analyze it yourself.

> -----Original Message-----
> From: Bob Wyman [mailto:[email-masked]]
> Sent: Saturday, May 31, 2003 7:16 AM
> To: Peter Kay; [email-masked]
> Subject: RE: [Asrg] TitanKey and "white lies"... (Faking SMTP hard
errors
> "improves" C/R utility?)
> 
> Peter Kay wrote:
> > I'll wait to see the responses and if positive,
> > I'll post off our Web site.
> 	If you're willing to post the data, I can't imagine why you
> would wait. This is a *research* group. *Any* real data is going to be
> welcomed by any rational participant -- especially if that data will,
as
> you suggest: "prove insightful". If you've got data, please post it!
> 
> 		bob wyman
> 
> 
> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Peter Kay
> Sent: Friday, May 30, 2003 8:06 PM
> To: Barry Shein; Scott Nelson
> Cc: [email-masked]
> Subject: RE: [Asrg] TitanKey and "white lies"... (Faking SMTP hard
> errors "improves" C/R utility?)
> 
> 
> Good news:
> 
> We've been "dogfooding" The Titan Key since around Jan of this year.
If
> the group is interested, we'd be happy to provide database logs of
spam
> "attacks" against email addresses that return a 550. I can't guarantee
> all the data is "there" but it will at least prove insightful.
> 
> I'll wait to see the responses and if positive, I'll post off our Web
> site.
> 
> Peter Kay
> President
> TitanKey Software
> direct: 808-377-9221.  Web: www.titankey.com
> The only technology that stops spam BEFORE it's even sent
> 
> 
>

---

From: "Eric D. Williams" <[email-masked]>
Date: Mon, 2 Jun 2003 22:44:03 -0400

On Sunday, June 01, 2003 6:21 PM, mathew [SMTP:[email-masked]] wrote:
8<...>8 for brevity only.
>
> 1. I send mail to Joe Smith.
> 2. I get a bounce message saying that there's no Joe Smith account, and
> that
>     my e-mail wasn't delivered. I delete it immediately, of course.
8<...>8

If your eMail server responds that a user does not exist, that is the proper 
response.  The proper response should be that the status of the delivery is 
'pending some disposition' or 'policy determination'.  That is the argument 
that it seems is at the root of this issue.  I did respond to a challenge from 
the system when I sent a previous eMail.  I may have recieved a "no such user" 
response in-band to the 'converstation'.

I may have deleted it as you suggest however.  In any event the only reason I 
did respond is that I knew that a message had been sent directly to that 
recipient, so you point is _very_ valid.

-e

---

From: "Peter Kay" <[email-masked]>
Date: Mon, 2 Jun 2003 20:32:01 -1000

OK, several requests are good enough for me. Data is posted.

There are 3 different spreadsheets at:

www.titankey.com/misc/logs/logScot.xls
www.titankey.com/misc/logs/logLes.xls
www.titankey.com/misc/logs/logPeter.xls

All 3 accounts are protected by The Titan Key. "Scot" and "Les" are
accounts that have been inactive for over a year and recently revived
for testing purposes.  The "peter" account is actually my own personal
older account.

Each spreadsheet contains several tabs as follows:

"Chart" is a chart of the attacks for the past 60 with a linear
regression trendline across the series.
"ChartData" contains a table of the values and labels to draw the chart
"PivotTable" is a pivotTable of the raw data used to collate data by the
day of the year
"Raw Data" is report of the  date, FROM address, To address, status
(internal), and day of year

I've purposely hidden the email "TO" addresses for my own personal
account as it is still active and there are privacy issues.  Scot and
Les have no privacy issues at this point as they have been inactive for
over a year.

I'm not a scientist nor a mathematician. My observations are only one of
a layperson that likes messing around with excel. I say the results are
officially "inconclusive". Scot trendline is going down slightly. Les'
trendline is going up. My trendline is  going down slightly.

Perhaps we can have the mathematicians and scientists provide qualified
analysis. If it would help,I can provide additional data such as IP
addresses and client hostnames.

Let me know what you all think.

Peter

errors
as
If
spam

---

From: "Eric D. Williams" <[email-masked]>
Date: Tue, 3 Jun 2003 22:07:28 -0400

On Tuesday, June 03, 2003 8:50 AM, Vernon Schryver 
[SMTP:[email-masked]] wrote:
8<...>8
> A "scaling problem" is one that gets worse with the growth of the net.
> The host table had a scaling problem.  The size of a default-free
> routing table may have a scaling problem.  Spam that grows with the
> number of spammers on the net is a "scaling problem."

Side note (again ;) - This is an interesting assertion.  Does 'spam' increase 
in proportion to the number of 'spammers'? Sounds like a "MAY", I think that is 
possible.  Does 'spam' increase in proportion to the number of recipients [read 
people eyes] accessible via the message transfer system, include here your 
favorite messaging technology, e.g. IM, SMS, Rim Pager, Palm?  Sounds like a 
strong case could be made, "SHOULD".

But the basis question is what is being 'scaled' the number of messages 
impacting recipients, the number of messages impacting the MTS or both? AND, is 
this a result or 1) the number of 'spamming' systems or 2) the number of 
'spammers' or 3) an relative decrease in the viable 'spam space' - valid 
recipient addresses available to some relatively constant pool of 'spammers'?

I only bring this up so that each of you may clarify your assumptions on the 
scaling issue.  Additionally, when I make reference to a decreasing viable 
'spam space' that is not an empirical or scientific assertion, but an 
assumption on 'spammer' practices.

Please clarify the assumptions relative to the 'scaling' issue, e.g. "Spam that 
grows with the number of spammers on the net" v. Spam that grows with the 
adaptability of address 'scapers'; Spam that grows with the number of 'known' 
legitimate mailboxes accessible to 'scrapers'; Spam that grows ...

Perhaps an analysis of these assumptions can provide some insight on what 
methodologies may be applicable to the reduction of 'spam' by imposing 
counter-measures against growth factors, e.g. poisoning the growth medium (I 
think that was the genesis of this thread).

Thanks,

-e

---

From: Scott Nelson <[email-masked]>
Date: Wed, 04 Jun 2003 09:31:26 -0700

At 06:50 AM 6/3/03 -0600, Vernon Schryver wrote:
>> From: Scott Nelson <[email-masked]>
>
>> ...
>> >I'm not saying it can't happen, but
>> >asking for a better reason than "it would be nice" and "maybe someday
>> >it will help somewhat."
>>
>> Why do you want a better reason?  
>
>yes, that's what I asked.
>

I didn't ask "Why, do you want a better reason?",
I asked, "Why do you want a better reason?"

Perhaps if I phrased it thus;

"Why are asking for an explanation of hope?"

Before I launch into a bizarre and probably hopeless off topic tangent
explaining why hope hasn't been crushed out of me even after many years 
of having my suggestions ignored, I'd like some assurance that I'm not just
confused as to what it was you were asking.  

>> Do you think the idea has merit?
>> Is it worthy of further investigation?
>
>If the idea can be specified and deployed before spam melts the net,
>it's worthwhile.  Otherewise, it's irrelevant in an anti-spam research
>group.  My responsibility for your idea stops with my pointing out
>problems that have apparently been overlooked.
>

You're not even responsible for that, and I am glad that 
you're willing to spend the time and effort to do it.
I might quibble about the style, but the function you are performing 
is IMO, vital, and I think you are doing an impressive job.

>
>> Would implementing it cause problems for the mail system in general?
>
>According to my rule book "why not?" is never a design justification
>and "nice" and "neat" are pejorative words when applied to a design
>feature.  If "Will it do any harm?" is not a form of "why not?",
>"nice," or "neat," it's not directly relevant to "will it stop spam
>in the lifetime of the net?"  Is your purpose in advancing the idea
>to fix the spam problem?
>
>

I've always assumed that it's sufficient that the balance
of good outweigh the balance of harm.  In other words,
I try to evaluate the cost/benefit of the suggestion.

The ideas were suggested as a possible solution to a problem 
that was raised regarding challenges - that most people have
trouble remembering what email they've sent, and that spammers
might therefore take advantage of the challenge vector to send more spam.

Since the suggestions are part of a larger system of changes 
involving challenge response, the "good" is dependant on how 
wide spread C/R becomes.  However, there is some good even 
without that, as I've already explained.  There might be more,
but it seems likely that those are going to be the biggest.

The next logical question from my perspective is 
"how much will it cost" (what's the exact amount of harm).

All changes have a cost, just by virtue of being changes.
(yet another way of saying "why not? isn't good enough".)
The cost here would be smaller than a normal change, 
since it's a proposed change to a proposed change, 
but I'm painfully aware that there will be an implementation cost.  
However, I'm also aware that it's easy for people (especially me) 
to overlook /other/ costs.
In particular, I wonder if there's some unobvious reason why
including messages-ids in a "in-reply-to:" header for a DSN is bad,
would damage some part of the mail system, not work over UUCP,
or ... well if I knew what else, I wouldn't need to ask.
Learning about these unknown costs is the most important from 
my perspective.  The cost of change is much easier to estimate, 
since change has been tried many times so there's a lot of history 
to compare against.

>> >> All these things are changes, and changes take time.
>> >> But the costs are small, and the potential benefits large.
>> ...
>
>> >realized there are two kinds of changes, those that I can make personally
>> >or that can be made by people I know and can convince, and very long
>> >shots that are almost always wastes of time to even talk about except
>> >while "shooting the bull."  
>>
>> Is there some other topic you feel is more worthy of discussion?
>
>Are we shooting the bull or looking for spam solutions that will have
>significant effects with 5 years?  
>

Both I think, and a lot of other things too.
Volunteer resources aren't limited in the same way as paid ones.
Having someone work on C/R doesn't detract from work on a taxonomy
which doesn't detract from work on RMX.  There might be a few people 
who are deciding what to work on based on the general consensus of
value, but mostly people only work on things they personally feel
are a good idea.  Convincing someone that their pet solution 
is a bad idea is more likely to result in them just going away 
than in them working on something else.  Unless they're pet idea 
is actually harmful, it's usually better to encourage them in their
folly than try and stop them.

>
>That's a serious question, because
>it seems almost all of proposals made in this list are what Barry
>Shein meant by testosterone ranting.  
>

As Theodore Sturgeon remarked, 90% of everything is crud.
This group doesn't appear to be an exception, 
but that doesn't mean there are no useful proposals being made.

>If this mailing list were about
>stopping spam in our lifetimes, then not only would there be no
>complaints that I'm not being "constructive" by pointing out lacks of
>likely effects, obvious gotchas, irrelevance (e.g. measurements of
>what spammers do today but could easily change tomorrow), but people
>would self-censor ideas that will have no significant effects.  On
>the other hand, if this mailing list is about proving that we each
>understand a subject that the general public thinks is very mysterious
>and quite important, then any sort of criticism is wrong.
>
>"Research" is not a "bull session," no matter what TV and movie writers
>might think.
>

Don't kid yourself, there would be complaints about your
behavior even if you were handing out twenty dollar bills and
telling everyone how nice they were.  People always want more, 
and complaints about not getting it seem inevitable to me.
Likewise, there will always be a broad spectrum of the clueless,
the hand wavers, the speech writers, and the chest thumpers in
any public group.

>
>> ...
>> >Yes, and C/R systems are unlikely to ever be widespread.
>>
>> I prefer to discuss the problems they can generate and potential 
>> changes that lessen the impact of those problems, 
>> even if the probability of wide spread deployment is remote.
>> Talk is cheap, and I prefer to try the cheap stuff first.
>
>That talk is cheap is the fatal problem with most talk about spam
>solutions.
>
>Your notion of Message-IDs could easily be written as an "Individual
>Submission" for an "Informational" RFC.  It would be more interesting
>that most RFCs in the RFC Index, but in 5 years I doubt 1% of mail
>would carry your Message-IDs.  Have you noticed the reference to
>"Recommendations for generating Message IDs" in RFC 2919?  How does
>your proposal differ from draft-ietf-usefor-message-id-01.txt?
>http://www.landfield.com/usefor/drafts/draft-ietf-usefor-message-id-01.txt
>seems to be an outlaw copy.  Notice its date.
>
>(By IETF canon law, Internet-Drafts "expire" or go down the memory
>hole after 6 months.)
>
>

I hadn't, but I have now that you've pointed it out.

The major difference is that my suggestions are in response
to a perceived problem, rather than a "wouldn't it be great if"
scenario.  I wouldn't be surprised though if no one used them
until after spammers actually started counterfeiting challenges,
and then counterfeiting message-ids.  Yes, I said I /hoped/ people
would adopt them much sooner than that, but with the possible exception
of challenge response systems themselves, 
that's not what I /expect/ to happen.

>> >Practically no one's correspondents increase with the size of the
>> >Internet (at least not this century).  
>> >
>> Quibble:  Most models I've seen suggest that the number of 
>> interactions increases as the cost of communication decreases, 
>> but it would be considerably more modest growth.
>

Yes, and the best models I've seen claim the number of communications 
that are made increases as N log(N) of the number of nodes that
one might wish to communicate with, which means even normal communication
has a theoretical scaling problem as cost approaches zero, 
and the size of the net approaches infinity.

But that much more modest exponential growth doesn't compare to
the spam problem, which is growing at close to Moore's law rates.  
That's why I qualified it with "quibble".

Scott Nelson <[email-masked]>
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg