Subject: Re: [Asrg] Turing Test ...

From: Art Pollard <[email-masked]>
Date: Wed, 02 Apr 2003 20:44:25 -0700

At 06:48 PM 4/2/2003 -0800, you wrote:
>On Wed, Apr 02, 2003, Art Pollard wrote:
> >
> > Why not allow the e-mail client to pose a Turing test to anybody that is
> > not in its whitelist?
>
>Depends on your whitelist.
>
>A group of people is supporting some open software so we get a lot
>of questions via e-mail. A reply can come from various sources.
>Which of the sources do you whitelist? Whenever I get a "TMDA"(?)
>challenge as a response to my reply I delete it. If I knew in advance
>that the person who asked the question uses such software I wouldn't
>waste my time to provide free support for them.

True.

However, I would imagine an intelligent client would whitelist

1) Addresses who have already passed the Turing test.
2) Addresses to which messages had been sent.
3) Domains to which messages had been sent for a limited period of time.

In the case you described, they sent you a support question and thus you 
would already be whitelisted for your reply.

In fact, you could even go so far as to whitelist hosts for a certain 
period of time.  So, for example, if I sent an e-mail to [email-masked], 
all of hp.com would be whitelisted for up to N days to facilitate a 
reply.  Once a reply had been made, the whitelist could be narrowed down to 
whoever sent the reply.

I believe there are some simple rules that the client could use in managing 
the whitelists that would significantly reduce the number of Turing tests 
that were presented to the senders of messages.

-Art
-- 
Art Pollard
http://www.lextek.com/
Suppliers of High Performance Text Retrieval Engines.

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Claus Assmann <[email-masked]>
Date: Wed, 2 Apr 2003 20:03:36 -0800

> would already be whitelisted for your reply.

No.

Maybe I should make it more clear: you send a question to sendmail.org,
you can get an answer from esmtp.org, some university, some Unix
vendor, some other domain...

Hence your original mail must already include a "token" which allows
the reply to go through without depending on the sender address
(domain). The simplest way would be to allow mail with

Subject: Re: your original subject

through... (hence "your original subject" should be a bit more
descriptive than "Hi" or "Help"...)
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Art Pollard <[email-masked]>
Date: Wed, 02 Apr 2003 22:43:00 -0700

>descriptive than "Hi" or "Help"...)

Oh, I got you.  And yes, you have a good point.

This could simply be worked around by having an additional header line such as:

UserID: ArtPollard_3448558020

This user ID could then be added / used in all the e-mail clients that I use.

Then when the whitelister whitelisted me for passing the Turing test it 
could key off of that in addition to my regular e-mail address.

It does become more complicated when I send a letter to you at:

[email-masked]

and my mail client automatically whitelists: [email-masked] but you 
choose to reply from a different e-mail address.  Even so, I think that 
this could reasonably be able to be worked around in most situations.  (For 
example, the address: [email-masked], could be added to the other header 
line for the whitelister to key off of in addition to the address you 
replied from).

Furthermore, I don't think that this would be / is the norm.  I think most 
people have a primary e-mail address they tend to work through.  True, at 
one point, I had about 11 different e-mail addresses but I typically only 
worked through one or two of them.

-Art

---

From: Vernon Schryver <[email-masked]>
Date: Thu, 3 Apr 2003 07:59:33 -0700 (MST)

> From: Art Pollard <[email-masked]>

> ...

> From: Art Pollard <[email-masked]>

> ...
> A properly designed system would have IMHO added you to the whitelist the 
> moment that the student you mentioned sent you a message. ...

That is not a good assumption if you look under the covers.  My case
is not uncommon, albeit less common than it was 10 or 15 years ago.
As stranger is likely to send mail to [email-masked] but receive a
rseponse from from [email-masked].

> ...
> If there were a standard way to do this sort of thing, even while you were 
> composing the message, your e-mail client could download the challenge 
> graphic and display it in your message composer. ...

> One of the things I am noticing on this list is that there are a number of 
> ways that we can solve this problem.  However, there seems to be resistance 
> to solving the problem simply because it makes e-mail more complicated in 
> some way. 

I don't agree.  Instead I see resistance to solutions that make false
or dubious simplifying assumptions.  The clearest example is the assumption
that if we hold all hold our breath, at least 80% of the net or
400,000,0000 people will change instantly their computers and habits.

Other assumptions that are not necessarily true are:

  - almost everyone uses a single mailbox, and sends from it.

  - strangers who don't speak English will be able to understand your
     challenge.  

  - strangers who speak English but are not technically inclined will
     understand or be willing to comply with such a frightening thing
     (that's not sarcasm.  Many people are literally frightened by 
     doing new things with computers.)

  - strangers will remember having sent you a message and so answer
     a challenge instead of assuming that you are just probing for
     valid addresses to spam.

  - it makes sense to talk about "downloading" a challenge/response
    graphic.  (think about those who don't use POP, IMAP, or a
    webmail scheme)

  - strangers will be willing to pay the costs of downloading 5 KBytes
     (or more?) of a graphic challenge.

  - strangers will value protecting your mailbox from spam enough
     to deal with your challenge/response.

> I have to wonder, what is easier filling out a challenge / response dialog 
> in your e-mail client for the several messages that you send per-day to 
> people who you have never e-mailed before (all the other people you have 
> communicated with have whitelisted you automatically) or throwing out 70+ 
> spam messages a day.

That makes the false assumption that when other people fill out a
challenge/response dialog, they avoid spam.  

It also assumes that manually throwing out 70+ spam messages/day cannot
be avoided except by such a system.  That is certainly false today.
My personal filters discard more than 70 messages/day without asking
me, and with a less than 0.1% false positive rate (for any reasonable
definition of false positive).

It also assumes that most users receive more than 70 or even more than
10 or 20 spam/day.  My guess (we need numbers) is that most users receive
fewer than 15 messages total per day and fewer than a dozen spam.

> No matter what solution we end up choosing, it is going to be more 
> inconvenient than is currently the case.

That is yet another dubious but not necessarily simplifying assumption.

Vernon Schryver    [email-masked]
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: wayne <[email-masked]>
Date: 06 Apr 2003 10:35:56 -0500

In <[email-masked]> Claus Assmann <[email-masked]> writes:

> (domain).

Hey, I have an idea!

You want to identify the message, so this "token" should be placed in
a header called, uh, how about "Message-ID:".  In order to make this
message-id work, you would want to make sure that it is unique to
every message that you send.

Then, when someone replies to this message the MUA could create a new
header, uh, let's call it "In-Reply-To:" that contains this
message-id.  That way we know that the reply was related to a message
that we sent.  Since the message-id is unique, it will be hard for a
spammer to guess and you could decide how long keep track of
message-ids so even if a spammer got ahold of one, it couldn't be used
for long.

The big plus of having things like a "message-id" and "in-reply-to"
headers would be for mailing lists like this.  A mail reader could tie
all the messages together into threads making it much easier to follow
the discussions.  To help with making the threading of messages more
robust, you could add a header that lists all the message-ids of all
messages involved in this set of branches of a thread.  Maybe we could
call this header something like "References:".

Wouldn't it be great if something like this had been created become
standard decades ago, so that even MicroSoft and Window mail systems
would follow such a standard?  (Hint to Phillip Hallam-Baker, Eric
D. Williams, etc.)  Or, at least didn't create damaged In-Reply-To
headers by breaking the message-id into two parts.  (Hint to Art
Pollard)

-wayne

---

From: J C Lawrence <[email-masked]>
Date: Sun, 06 Apr 2003 13:57:50 -0700

On 06 Apr 2003 10:35:56 -0500 

>> Maybe I should make it more clear: you send a question to
>> sendmail.org, you can get an answer from esmtp.org, some university,
>> some Unix vendor, some other domain...

> every message that you send.

Filtering returning bounces based on Message-ID requires that the MX
keep a DB of known-valid IDs.  This is a non-trivial expense.  There are
other problems related to secondary MXes, load sharing, architectures
with multiple ingress and egress paths, disjoint arrangements (satellite
operations which handle their own deliveries but mail is routed
centrally) etc.

-- 
J C Lawrence                
---------(*)                Satan, oscillate my metallic sonatas. 
[email-masked]               He lived as a devil, eh?		  
http://www.kanga.nu/~claw/  Evil is a name of a foeman, as I live.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: wayne <[email-masked]>
Date: 06 Apr 2003 19:12:21 -0500

In <[email-masked]> J C Lawrence <[email-masked]> writes:

> keep a DB of known-valid IDs.  This is a non-trivial expense.

I believe that the expense is usually trivial compared with the cost
of the time that spam places on the human.  There are a lot of people
who seem to be penny wise and pound foolish about spam.

I'm not saying that every technique used to reduce spam is worth
whatever costs may be needed to implement it in any given situation.
However, if you find that many of the techniques are costly because of
how you have designed your mail system, you may be better off
rethinking how your mail system is designed.

-wayne

---

From: J C Lawrence <[email-masked]>
Date: Sun, 06 Apr 2003 18:29:46 -0700

On 06 Apr 2003 19:12:21 -0500

> who seem to be penny wise and pound foolish about spam.

I'm not so sure.  In the last year my domain has received a few hundred
thousand (I don't have an accurate count, I know I just went thru 80,000
a few weeks ago, and that there were three other cases in the last year
at roughly a tenth that size and similar behaviour for other
non-existent accounts.

Am I missing something here, is abuse via bounces really common and
growing?

> rethinking how your mail system is designed.

Fair dinkum.

ObNote: There are some issues here with Return-Paths which don't point
back to the Sender, but they can be handled.

---

From: Art Pollard <[email-masked]>
Date: Mon, 07 Apr 2003 12:05:52 -0600

True.

There might be some ways to handle this using cryptographic techniques.
I am a bit rusty here so excuse any errors in logic here but here is what I 
was thinking...

After the Turing test or similar authorization process, you could exchange 
the information needed for using / testing digital signatures.

Then when any additional messages are sent, the new messages would have a 
signature line.  You could not only ensure that the person you received the 
message from was "authorized" (as in the sense that one would have in 
checking addresses) but also is who they say they are.

This would also help solve the problem of people with multiple e-mail 
addresses.  They could have lots of e-mail addresses and as long as they 
had the proper signature set up at the different e-mail addresses / 
accounts then the fact that they had different addresses could be entirely 
transparent.

So, the any approvals and whitelisting that is done need not be done on the 
originating e-mail address but could also be done on the digital signature.

Now, I am starting to get a budding idea here.....

What if..... A new digital signature were issued to each new person who you 
correspond with.  Then if you start getting spam from that person / 
organization, you could simply disapprove them.  This would get around any 
possibility of people sharing information to try to get around the 
restrictions.

-Art

---

From: J C Lawrence <[email-masked]>
Date: Mon, 07 Apr 2003 12:27:38 -0700

On Mon, 07 Apr 2003 12:05:52 -0600 
Art Pollard <[email-masked]> wrote:

> After the Turing test or similar authorization process, you could
> exchange the information needed for using / testing digital
> signatures.

> Then when any additional messages are sent, the new messages would
> have a signature line.  You could not only ensure that the person you
> received the message from was "authorized" (as in the sense that one
> would have in checking addresses) but also is who they say they are.

Absolutely, and this raises a critical distinction:

  To what degree is the problem to be handled at the MTA level, the LDA
  level or the MUA level?

To get the platitudes out of the way, earlier is generically better.
(Doh!)  My expectation is that we're going to have to involve all three
camps, heavily and discretely.  More specifically I expect that the
final strategies realised and deployed will be multiple and overlapping,
without consistent direction and design cooperation between the levels.

Consider the simple case of consent tokens:  

  Who generates the consent token?  Who tracks it?  Who checks it?  Who
  has audit oversight on the consent trail?

(I won't deal with consent acquisition at this point)

Crytpo techniques can certainly be applied to make processing of consent
tokens unidirectional (eg only one thing can generate them, any thing
may check their veracity).  They can also be applied to the ends of
arranging mutual consent from a contact, as well as forming the visible
expressions and encodings of consent.

> addresses.  They could have lots of e-mail addresses and as long as
> they had the proper signature set up at the different e-mail addresses
> / accounts then the fact that they had different addresses could be
> entirely transparent.

<nods>

This is an interesting one -- the ability to make different addressing
forms invoke the same consent methods, or for them to be externally
resolvable as representing the same human -- but outside of the nice
efficiency aspects I'm not clear that its our business (yet).

> Now, I am starting to get a budding idea here.....

Oh dear.

> What if..... A new digital signature were issued to each new person
> who you correspond with.  Then if you start getting spam from that
> person / organization, you could simply disapprove them.  This would
> get around any possibility of people sharing information to try to get
> around the restrictions.

Aye, I proposed something of this when I talked about forward chained
Received: headers, to whit the use of TMDA-style sender addresses as an
expression of consent tokens. This could be expressed thru overloading
the SMTP envelope.  Consider:

  Bubba emails Boffo.

    From: bubba@here
    To: boffo@there
    Subject: Hi!

    ...etc...

  Assuming a consent negotiation has previously occurred Bubba's MUA
  tags an envelope on his message ala:

    Return-Path: bubba+deadbeef111abcdef978@here
    Envelope-To: boffo+a90f234af57eee8bacab@there

  Wherein the plus addressing tokens represent the consent tokens.  The
  payload, the message itself, is left unchanged.  More specifically:

    Bubba's token of "deadbeef111abcdef978"is opaque to Bubba, but
    encodes the following information for Bubba:

      This is one of my consent tokens, and not something someone
      forged.

      This consent token is for the exclusive use of bubba@there, and
      use with any other sender address will fail.

    But for Boffo, Bubba's token is compleatly opaque.  Its merely a
    string which Boffo's MUA knows to use with Boffo.

    Similarly for Boffo's token of "a90f234af57eee8bacab" as regards
    what it means for Boffo and Bubba.
  
  Impacts at the server-level:

    Everybody has to support plus-addressing.  

      ObNote: There needs be no standardisation on the plus addressing
      character used ('-' vs '+' etc): the transform required of an
      address to add a consent token can be stated/agreed-upon/etc at
      consent negotiation/grant time.

    Future servers and LDAs can add support for storing secrets for
    processing consent tokens at SMTP or LDA time.  THis is not required
    for deployment however.

  Niceties:  

    Plus-addressing is not exposed to the humans, just the transport
    layers (and perhaps some headers for better tracking/auditing).

    The addresses humans use don't need any change.  They never see or
    need to type the consent tokens.

    Software, and in particular the MUAs do all the token tracking,
    application, checking etc, so MUAs can revoke any previously granted
    token as well.  Humans don't need to.  

    If the secret required for encoding the consent token is shared with
    upstream MTAs, they can execute the consent token checks (not
    revoked) at receipt time.  Or, for more slightly more paranoid
    arrangements, the secret may be shared only with the LDA.

Now methods and means of establishing consent tokens, of the
negotiations required or a priori contacts or data are an entirely
different matter.

---

From: wayne <[email-masked]>
Date: 07 Apr 2003 16:19:40 -0500

In <[email-masked]> J C Lawrence <[email-masked]> writes:

> [ ... ] 

Well, the bounces really are annoying to people who have their email
addresses forged by spammers.

However, there is another use for this kind of database of all known
message-IDs that have been (recently) sent.  Right now, spammers can
use a MAIL FROM:<> in order to escape some verification techniques.

-wayne

---

From: [email-masked]
Date: Wed, 16 Apr 2003 08:54:47 -0700

It seems like the plus addresses show up in the RCPT TO envelope line
and may not show up anywhere else. (That is when the TO is mailing
list, or when a BCC is used.)

Can we be sure that plus address will be visible to the MUA?

/Russell

      Niceties:

---

From: J C Lawrence <[email-masked]>
Date: Wed, 16 Apr 2003 09:51:50 -0700

On Wed, 16 Apr 2003 08:54:47 -0700 

> list, or when a BCC is used.)

As far as the received message at that time I was envisioning the plus
addresses showing up in Sender:, Envelope-to:, and Return-Path:.  

> Can we be sure that plus address will be visible to the MUA?

No, since then I've done a little research.  Several MTAs rewrite
Sender:, and the othe two headers are not reliably/widely supported.
Methinks we'll have to expose the consent token in the from:/To:
headers, embed it in the Message-ID, and potentially think about use of
the Subject header for control messages.

---

From: "Dave Lampert" <[email-masked]>
Date: Sat, 19 Apr 2003 01:00:05 -0400

J.C. and Art,

We may all be talking about the same idea... see also:

[Asrg] Gray list: rating of associates
https://www1.ietf.org/mail-archive/working-groups/asrg/current/msg01552.html

Instead of using a plus-type email like "boffo+a90f234af57eee8bacab@there",
it uses a GUID like the one I'm sending my email from now.

I use this system now and it works whether other servers recognize this new
approach or not. Every email address is unique to the sender-receiver
pairing.

Dave

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of J C
Lawrence
Sent: Monday, April 07, 2003 3:28 PM
To: Art Pollard
Cc: [email-masked]
Subject: Re: [Asrg] Turing Test ...

<nods>

Oh dear.

  Bubba emails Boffo.

    From: bubba@here
    To: boffo@there
    Subject: Hi!

    ...etc...

  Niceties:

---

From: J C Lawrence <[email-masked]>
Date: Sat, 19 Apr 2003 00:21:10 -0700

On Sat, 19 Apr 2003 01:00:05 -0400 

> We may all be talking about the same idea... see also:

Not quite.  The proposal I'm writing for posting later this week
(current consulting permitting) is at its heart a protocol for consent
token manipulation, along with a set of suggested token types and a
possible model of how tokens supporting such __could__ be implemented.
The central expectation is that implementations at the MUA level will
vary extensively in both detail, and feature-set -- which is as it
should be.

> https://www1.ietf.org/mail-archive/working-groups/asrg/current/msg01552.html

I'll ignore the patent aspects beyond the notes that: 

  a)I consider them patently ludicrous (pun intended)

and:

  b) should they be enforced to any trivial degree; automatic and
  instant cause for the proposal to be discarded with prejudice.

> Instead of using a plus-type email like
> "boffo+a90f234af57eee8bacab@there", it uses a GUID like the one I'm
> sending my email from now.

There is nothing unique or interesting to a plus address, and in fact
there is nothing I've seen that you are doing which is particularly
different from a plus address.  You have a way of taking a standard
"base address" and extending it with additional data, or taking such an
extended address and deriving from it a "base address".  The fact of
what the extracted token is used for, or what data it represnts or its
other semantics are quite arbitrary and largely irrelevant.  Those are
the exact same operations that plus addressing supports.  The choice of
character or regex used is utterly arbitrary and quite unagreed upon in
the field.

> I use this system now and it works whether other servers recognize
> this new approach or not. Every email address is unique to the
> sender-receiver pairing.

Sender consent tokens (using the TMDA definitions for convenience) are
only one form of consent token I'm looking to suggest.  I've little
doubt that should this be adopted that most implementations won't do my
full set, and that a few will exceed it.  <shrug> Which is as it should
be.  Tokens should be opaque to all but their generators.

---

From: mathew <[email-masked]>
Date: Sun, 20 Apr 2003 17:36:29 -0400

On Saturday, Apr 19, 2003, at 01:00 US/Eastern, Dave Lampert 
<[email-masked]> wrote:
> this new

Umm, yeah. So how large are your business cards? And do you expect 
people to type something like 
[email-masked]
to first get in touch with you?

mathew