Subject: PRA algorithm and use of non-standard header fields

From: Roy Badami <[email-masked]>
Date: Fri, 16 Jul 2004 02:20:50 +0100

I realize that new docs are in preparation, but I'd like to raise this
issue anyway...

A few people (myself included) have raised concerns about the
dependence of the PRA algorithm on non-standard header
fields.

Having thought about this a bit more, I'd like to air my specific
concerns about the PRA algorithm as defined in -marid-core-00 and -01

1) I really don't like the idea that the semantics of a standard IETF
   protocol will depend on the presence and contents of non-standard
   fields that are defined nowhere (other than in specific MTA
   documentation).

2) Because the names of these fields are fairly obvious and natural
   names, and not obviously MTA-specific, it seems entirely plausible
   to me that some lesser known or custom mail systems may use these
   fields in a way that is somewhat different to the ways in which the
   designers of the algorithm invisaged.

2) I liked the original Caller ID PRA algorithm; it was intuitive.
   Ignoring corner cases of ill-formed messages for a moment, it
   corresponds *exactly* to what I have always done to determine who
   sent me a message (ie the person who pushed the button on the MUA,
   or the mailing list that redistributed the message).  That is how I
   would read the headers, based on my understanding of RFC822 and
   RFC2822, to determine who or what sent me the message.

   Granted, MARID doesn't want to know who pushed the button, it wants
   to know who initiated the SMTP transaction, and forwarders make a
   difference there. But the Caller ID algorithm is one that everyone
   who knows how to read (2)822 headers is already familliar with; the
   revised algorithm in marid-core is not.

3) I don't think the currently proposed algorithm works.  Tony Finch
   raised a concern about the semantics of this some time ago, but I
   don't think this was ever followed up on.  I'd like to raise some
   more detailed concerns, having given the matter further thought.

   Consider these two scenarios:

   a) A message is sent, and it transits one or more MTAs that add
      Delivered-To.  The initial recipient then resends the message to
      a second recipient using their MUAs resend functionality.

      In this case step 1 of the PRA algorithm will correctly select
      the PRA from the Resent-* headers.

   b) Consider I ([email-masked]) have a message in my inbox.  For sake of
      argument, assume it currently contains no Resent-* headers or
      Delivered-To etc.

      Assume I now use it to resend the message to [email-masked], and that
      account is configured to forward it to [email-masked].  Assume that the
      MTA at b.com adds

	  Delivered-To: [email-masked]

      Assume the MTA at c.com wishes to perform MARID checks.

      In this case, the PRA algorithm will pick the pre-existing
      Resent-* header in preference to the more recent Delivered-To
      header that was added by b.com.  So c.com will incorrectly
      determine the PRA as [email-masked] instead of [email-masked]

   So the fact that some MTAs already add these headers when
   forwarding doesn't really help us, at least with the PRA algorithm
   in its current form.  To make things work correctly for forwarded
   messages that *already* contain Resent-* headers still requires
   modifications to forwarders.

   What you need to do to fix this is to take whichever Resent-* or
   Delivered-To header that was most recently added to the message;
   however I fear determining this reliably will involve making far
   too many assumptions about exactly how MTAs behave when adding
   these non-standard fields.

4) It complicates post-SMTP-time analysis tools.  It's my
   understanding that MTAs that use Delivered-To, etc will add these
   headers at final delivery as well as when forwarding.  Hence if a
   tool wants to determine the PRA of a message after delivery, and
   that tool is running on a site where the delivering MTA adds
   Delivered-To, then the tool needs to know to ignore the
   locally-added Delivered-To field when computing the PRA.

5) A corrollary of 4.  For PRA-based MARID to be useful to the masses,
   we need to encourage MUA authors to display or highlight in some
   way the PRA to the end user, so that they know which ID has been
   authenticated without having to read and understand the 2822
   headers in their entirity.

   So the MUA needs to run the PRA algorithm.  As with 4 above,
   dealing with locally-added Delivered-To fields will complicate
   this process.

So I'd like to suggest that step 3 simply be dropped from the
marid-core-01 PRA algorithm.  IMHO step 3 turns a clean algorithm into
a messy heuristic, and in its current form has serious problems, not
all of which can easily be fixed.

	      -roy

---

From: Mark Lentczner <[email-masked]>
Date: Thu, 15 Jul 2004 19:55:45 -0700

> dependence of the PRA algorithm on non-standard header

If you are referring to the three headers, Delivered-To, X-Envelope-To 
and Envelope-To, in my last meeting with the authors of marid-core, it 
was agreed to remove them from the algorithm.  With reference to 
marid-core-01, you can consider step 3 in section 4 to be deleted.

We all agreed with your logic of using non-standard headers which could 
be shown to be implemented differently in different MTAs.  I showed the 
way Postfix uses Delivered-To as an existence proof of this problem.

	- Mark

---

From: wayne <[email-masked]>
Date: Thu, 15 Jul 2004 22:23:13 -0500

> marid-core-01, you can consider step 3 in section 4 to be deleted.

How much does the removal of these headers change the effectiveness
and false positive rate of the PRA algorithm?

-wayne

---

From: Mark Lentczner <[email-masked]>
Date: Thu, 15 Jul 2004 20:39:00 -0700

I wrote:
> and Envelope-To, ... it was agreed to remove them ...

> and false positive rate of the PRA algorithm?

I don't know.  I haven't seen any data on the effectiveness or false 
positive rate of the PRA algorithm.  I haven't seen any data on it at 
all, but I'd sure like to.

Does anyone have a largish data base of messages (just need the 
headers) categorized as spam/not-spam?  I would happy to code up a 
quick PRA test and run it over the dataset.  Of course, without the 
domains publishing records, we'll have to make some guesses as to if 
the identified PRA matches the smtp client IP...

	- Mark

Mark Lentczner
http://www.ozonehouse.com/mark/
[email-masked]

---

From: Roy Badami <[email-masked]>
Date: Fri, 16 Jul 2004 11:45:11 +0100

>>>>> "Mark" == Mark Lentczner <[email-masked]> writes:

    Mark> If you are referring to the three headers, Delivered-To,
    Mark> X-Envelope-To and Envelope-To, in my last meeting with the
    Mark> authors of marid-core, it was agreed to remove them from the
    Mark> algorithm.  With reference to marid-core-01, you can
    Mark> consider step 3 in section 4 to be deleted.

Thanks for the info.

    Mark> We all agreed with your logic of using non-standard headers
    Mark> which could be shown to be implemented differently in
    Mark> different MTAs.  I showed the way Postfix uses Delivered-To
    Mark> as an existence proof of this problem.

One of my problems here is that I'm not intimately familiar with any
MTA that adds these headers.  The only MTA I would claim to be
familiar with in any detail is sendmail, which uses none of these
headers.

In any case, that's by the by.  I'm happy that they're gone from the
spec.

Incidentally, one further very minor question I'd like to raise.  Is
it necessary to refer only to non-empty headers (ie to explicitly
ignore all empty headers)?  

I left the word "non-empty" in my PRA proposal because it was in the
original.  But none of the headers in question are allowed to be
empty, and it's an extra complication for implementations that may not
be necesary unless there are real concerns that empty header fields
occur significantly in the wild in otherwise well-formed messages.

	      -roy

---

From: Chuck Mead <[email-masked]>
Date: Fri, 16 Jul 2004 12:30:58 -0400

Mark Lentczner wrote:
|
| I wrote:
|
|> If you are referring to the three headers, Delivered-To, X-Envelope-To
|> and Envelope-To, ... it was agreed to remove them ...
|
|
| Wayne wrote:
|
|> How much does the removal of these headers change the effectiveness
|> and false positive rate of the PRA algorithm?
|
|
| I don't know.  I haven't seen any data on the effectiveness or false
| positive rate of the PRA algorithm.  I haven't seen any data on it at
| all, but I'd sure like to.
|
| Does anyone have a largish data base of messages (just need the headers)
| categorized as spam/not-spam?  I would happy to code up a quick PRA test
| and run it over the dataset.  Of course, without the domains publishing
| records, we'll have to make some guesses as to if the identified PRA
| matches the smtp client IP...

I have a large store of this kind of stuff (ham/spam) that I could tar
up and send to you... lemme know if you want it and where to send it,
or, if you prefer I could tar it and make it available for download.

- --
Chuck Mead
[email-masked]
Chief Tech @ http://moongroup.com - http://anirononline.com

---

From: Andrew Newton <[email-masked]>
Date: Fri, 16 Jul 2004 13:33:01 -0400

> the identified PRA matches the smtp client IP...

Without domains publishing records, the guesses would be pretty large.
I ran some tests back in May:
http://hxr.us/blojsom/blog/grumpops/computers/? 
permalink=00E5176444341F7E1A3946B676CED44E.txt&smm=y
and I believe the results to be inconclusive.

-andy

---

From: Andrew Newton <[email-masked]>
Date: Fri, 16 Jul 2004 13:41:13 -0400

> I ran some tests back in May:

Excuse my stupidity.  That wasn't a PRA test, just a 2822.From & 
MAILFROM test.
Sorry.
-andy

---

From: Andrew Newton <[email-masked]>
Date: Mon, 19 Jul 2004 12:13:07 -0400

> the identified PRA matches the smtp client IP...

I reran my tests using PRA and SPF-classic.  Here are the results:

HAM  - 15678 messages (most from mailing lists, btw)
====   Msgs w/usable RR     Deny     Pass
        ----------------     -----    -----
SPF-C               32%      3.9%    23.5%
PRA                 32%      4.5%    23.7%

SPAM - 15710 messages
====   Msgs w/usable RR     Deny     Pass
        ----------------     -----    -----
SPF-C              4.6%     74.1%     2.5%
PRA                6.5%     51.0%     1.7%

Other interesting data:  of the 31,388 messages, there was not a single 
one where SPF-C and PRA contradicted each other.  Also, of the 31,388 
messages,  2822.From was not the PRA in 14,945 or 48%, but of the 15678 
HAM messages the percentage is 78%.

The only conclusion that I can draw is this:  without more domains 
publishing records, there is no way to know if there is a difference 
between PRA and SPF-C.

I do have the following comments on the marid-core PRA algorithm:

1) The six-steps really ought to be put into pseudo-code with each step 
spelled out in a separate routine.  I found that the textual 
descriptions were a little confusing.  If need be, I can contribute my 
Python code.

2) Many of the steps talk about a "non-Empty" header.  Isn't this 
requirement also fulfilled by step 5, therefore making this requirement 
in the subsequent steps redundant?

3) It may be necessary to modify the check of the "Sender" header 
because many systems do not attach a domain name to a mailbox address 
if the injection and delivery are on the same box.  Or perhaps this 
should go into an applicability statement section 7 regarding checking 
of email local to the system.

-andy

---

From: Mark Lentczner <[email-masked]>
Date: Mon, 19 Jul 2004 14:32:22 -0700

Thanks, Andy for your stats.  I've been running my own this weekend, 
and I came up with slightly different results.   I started with four 
data sets: my inbox from 2003 (~1500 messages), my last three days of 
spam (~500 messages), and two from Chuck Mead (ham @ ~1000 messages and 
spam @ ~4000 messages).

I took messages, computed the PRA and compared it's domain to the 
domain of the envelope from.  If they were the same, I did no more 
checks, since they would use the same record and have the same results 
(barring records that make distinctions based on localparts.)  The 
percentage where they differ is:

	ham1  17%
	ham2  16%
	spam1  8%
	spam2  9%

I then ran the SPF record check against the PRA and env. MAIL FROM 
identities on these messages.  I considered both only messages where 
the SPF records were published, and all cases using guesses for the SPF 
records when missing.  Here is the percentage of time a guess was used:

	      PRA  FROM
	ham1  82%   92%
	ham2  72%   75%
	spam1 67%   94%
	spam2 78%   89%

Unlike Andy, the query results differed some percentage of the time:

	ham1  66%
	ham2  27%
	spam1 43%
	spam2 31%

Lastly, like Andy, I looked at the Pass and Fail percentages (for both 
only published and published + guessed records):

Ham1 Summary:               published SPF    + guessed SPF
-----------------------   ---------------  ---------------
   PRA Pass:                     11 ( 24%)        81 ( 31%)
   env. from Pass:               18 (100%)       241 ( 94%)

   PRA Fail:                      0 (  0%)         0 (  0%)
   env. from Fail:                0 (  0%)         0 (  0%)

Ham2 Summary:               published SPF    + guessed SPF
-----------------------   ---------------  ---------------
   PRA Pass:                     14 ( 53%)        62 ( 64%)
   env. from Pass:               16 ( 66%)        72 ( 75%)

   PRA Fail:                      7 ( 26%)         7 (  7%)
   env. from Fail:                0 (  0%)         0 (  0%)

Spam1 Summary:              published SPF    + guessed SPF
-----------------------   ---------------  ---------------
   PRA Pass:                      2 ( 16%)         6 ( 16%)
   env. from Pass:                0 (  0%)        10 ( 27%)

   PRA Fail:                      5 ( 41%)         5 ( 13%)
   env. from Fail:                2 (100%)         2 (  5%)

Spam2 Summary:              published SPF    + guessed SPF
-----------------------   ---------------  ---------------
   PRA Pass:                     40 ( 44%)        94 ( 22%)
   env. from Pass:                0 (  0%)        97 ( 23%)

   PRA Fail:                     22 ( 24%)        22 (  5%)
   env. from Fail:               13 ( 29%)        13 (  3%)

While the numbers are low, and I'd be first to worry about generalizing 
from these stats, I can begin to see a trend:

1) Over 80% of mail is not complicated and the PRA produces the same 
domain as env. from.  Oddly enough, even more so for spam.  (I suppose 
spammers just haven't caught on.)

2) While still only a small percentage of sites have records, more 
sites identified by PRA had records than env. from.  This could be 
because the PRA is more often a bigger site.

3) When the domains differ, the check result from PRA vs. env. from is 
often different.  This is the only stat. that is at odds with Andy's 
findings.

4) For ham, env. from produces a greater percentage of passes than PRA. 
  It also produces few fails, but the numbers here are very low.  For 
ham, more passes and fewer fails is good.

5) For spam, the numbers seem less conclusive, but PRA seems to produce 
more definite results (passes and fails) than env. from.

I've attached the detailed output of my script runs for anyone that 
wants to plunge in deeper...

	- Mark

Mark Lentczner

---

From: Mark Lentczner <[email-masked]>
Date: Mon, 19 Jul 2004 14:40:30 -0700

Andy wrote:
> 1) The six-steps really ought to be put into pseudo-code with each 
> step spelled out in a separate routine.  I found that the textual 
My stats also collected how often each step was used in yielding the 
result.  Out of over 7,000 messages, only 8 ever used step 2 
(Resent-From:) and none used step 1 (Resent-Sender:).  Perhaps these 
aren't important and only complicate the issue.  Without them, the 
"Sender if it is there, otherwise From" logic is just the 2822 
definition of who the injector is.

> requirement also fulfilled by step 5, therefore making this 
> requirement in the subsequent steps redundant?
I agree, though my implementation followed the text to the letter - 
(some of them didn't say non-empty, and I distinguished these cases.)  
I doubt it, not one header in my 7,000 messages was empty.

> of email local to the system.
In my analysis I immediately discarded all local messages, so I can't 
comment.  But this does bring up the question about what constitutes 
"malformed".  While 2822's intentions are pretty strict, it does seem 
to allow for the realities of decades of older mail with not-so-strict 
header formatting.  I'm not sure where to go on this, but I do realize 
that we only expect the PRA to be computed over newly created messages, 
that should, by now, conform to the stricter intentions of 2822.

	- Mark

---

From: Tony Finch <[email-masked]>
Date: Mon, 19 Jul 2004 23:06:41 +0100

On Mon, 19 Jul 2004, Mark Lentczner wrote:

> My stats also collected how often each step was used in yielding the result.
> Out of over 7,000 messages, only 8 ever used step 2 (Resent-From:) and none
> used step 1 (Resent-Sender:).  Perhaps these aren't important and only
> complicate the issue.  Without them, the "Sender if it is there, otherwise
> From" logic is just the 2822 definition of who the injector is.

No, you have to include the Resent- headers in the algorithm to be
properly conformant with RFC 2822 and RFC 822. One of the standard MUAs at
our site for over 10 years has been Pine which does roughly the right
thing with Resent- headers (it uses 822 rather than 2822 syntax in this
respect). It might be a bit rare but it isn't unimportant.

> But this does bring up the question about what constitutes "malformed".  While
> 2822's intentions are pretty strict, it does seem to allow for the realities
> of decades of older mail with not-so-strict header formatting.  I'm not sure
> where to go on this, but I do realize that we only expect the PRA to be
> computed over newly created messages, that should, by now, conform to the
> stricter intentions of 2822.

I recently experimented with header syntax verification as an anti-spam
measure, and I found out quite how atrociously bad modern software is at
following RFC 2822. Particularly bad examples include the Microsoft MUA
which produces headers like
	To: <Undisclosed-recipients:;>
or bare local parts in Sender: headers, or nothing but a display-name, or
the display-name following the angle-addr instead of coming before it, or
an unquoted addr-spec before an angle-addr. Utterly hopeless.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
NORTHWEST FITZROY SOLE: SOUTHERLY VEERING WESTERLY 5 OR 6. RAIN OR SHOWERS.
MODERATE OR GOOD, OCCASIONALLY POOR.

---

From: Roy Badami <[email-masked]>
Date: Mon, 19 Jul 2004 23:26:59 +0100

>>>>> "Andrew" == Andrew Newton <[email-masked]> writes:

    Andrew> 2) Many of the steps talk about a "non-Empty" header.
    Andrew> Isn't this requirement also fulfilled by step 5, therefore
    Andrew> making this requirement in the subsequent steps redundant?

No.  Consider an message that contains a valid From header and an
empty sender header.  Currently, step 3 will ignore the (illegal)
empty Sender header, and step 4 will select the From header.

If the words "non-empty" were removed, step 3 would select the empty
Sender header, and step 5 would then fail to determine a PRA because
the header was malformed.

Whether this case is important in practice, I don't know.  The current
PRA algorithm is based on a proposal I made to this WG in response to
-core-00.  I specifically specified it this way because the Caller ID
PRA algorithm considered only non-empty headers, and handling of empty
headers wasn't the issue I was concerned about.  (My proposal was
intended to be a "minimum change" proposal from the original
algorithm.)

I raised the question a week or so ago as to whether it might be an
idea to remove the words "non-empty".  There seems no pressing need
for them in the PRA algorithm as currently defined. It adds a (small)
amount of complexity, and only changes the outcome for ill-formed
messages.

The cost is small, and if there is a belief that spurious empty
headers occur in otherwise well-formed messages in the wild, then I'm
all for keeping the wording.  But since checking whether the headers
are empty is an extra conceptual step, if we retain it, we should at
least have some idea as to why we're doing this...

      -roy

---

From: Roy Badami <[email-masked]>
Date: Tue, 20 Jul 2004 00:05:49 +0100

>>>>> "Mark" == Mark Lentczner <[email-masked]> writes:
    Mark> My stats also collected how often each step was used in
    Mark> yielding the result.  Out of over 7,000 messages, only 8
    Mark> ever used step 2 (Resent-From:) and none used step 1
    Mark> (Resent-Sender:).  Perhaps these aren't important and only
    Mark> complicate the issue.  Without them, the "Sender if it is
    Mark> there, otherwise From" logic is just the 2822 definition of
    Mark> who the injector is.

I disagree.  The PRA algorithm, in its current form, is a perfect
description of the 2822 semantics.

Surely if the message contains a Resent-From (and no Resent-Sender),
then according to 2822 the Resent-From is the person who sent you this
message?

And if it contains Resent-From and Resent-Sender, then the
Resent-Sender is the person who sent you the message on behalf of the
Resent-From.

Personally I think this analysis of statistics is useful, but possibly
misguided in intent (or at least misconstrued in intent by some
participants).

I don't see the PRA algorithm as some heuristic to try and construct
some new kind of identity from the headers; the PRA is a natural
concept that falls out of the headers defined by 822.  If you want to
know who (according to the headers) sent you a message, would you do
anything substantially different from what the PRA algorithm
specifies?

If we're going to use a header identity rather than the envelope
identity, then the PRA is the only obvious identity to use; I've
believed that since before Caller ID was proposed, and I have yet to
see anyone make a convincing argument for any other header identity.

An IP-based authentication scheme obviously needs to look at who
actually sent the message, not on whose behalf it was sent (that
implies using Sender in preference to From).  And it needs to look at
the immediate sender, which implies using the Resent headers in the
case of a resent message.  And using the most recent resent headers in
the case of a multiply-resent message (which is permitted by 2822).

If we're going to spend time on the PRA, I think we should be
analysing it semantically, not statistically.  We should be looking at
the details and the corner cases.

How does the current algorithm handle borderline messages (ie
technically malformed, but acceptable to a liberal parser)?  Are there
any important corner cases that occur in the wild due to non-compliant
systems?

What are the issues with header order and the algorithm for selecting
the Resent header?  (822 restricted messages to a single set of Resent
headers, but places no restrictions on placement.  2822 contrains
placement and allows multiple resending, but qualifies this with a
confusing SHOULD in the description of Resent headers which seems to
be at odds with the BNF.)

We know the broad outline of what is going to go to last call.  Unless
anyone thinks that they can gain concensus for a significant change of
tack, lets focus on the little details.

I certainly wasn't expecting my proposal for revising the PRA
algorithm to be adopted with absolutely no discussion or even comment.
It was the product of several minutes of consideration on my part, and
was intended to be a starting point for discussion, not a fait
accomplis.  I think it works, and the -core authors have done a decent
job of restating it more succinctly, but have we got the semantics
exactly right?

I _think_ so (besides my doubts about retain the "non-empty" check)
but I don't have confidence that anyone other than myself and the ID
authors have spent any time looking at it with a critical eye.

This is really a general complaint about the way this WG has operated;
there has been _far_ too little technical discussion of the details of
the proposals that are supposed to be going to last call in a very
small number of weeks.  Lots of discussion of the big picture, at
times verging on out-of-scope, but little discussion of the actual
documents that we are supposed to be advancing...

(This complaint is in no way directed at Mark or any other specific
contributors to this thread, but to the WG as a whole.)

      -roy

---

From: Roy Badami <[email-masked]>
Date: Tue, 20 Jul 2004 00:32:53 +0100

>>>>> "Tony" == Tony Finch <[email-masked]> writes:
    Tony> No, you have to include the Resent- headers in the algorithm
    Tony> to be properly conformant with RFC 2822 and RFC 822. One of
    Tony> the standard MUAs at our site for over 10 years has been
    Tony> Pine which does roughly the right thing with Resent- headers
    Tony> (it uses 822 rather than 2822 syntax in this respect). It
    Tony> might be a bit rare but it isn't unimportant.

VM (for EMACS) uses Resent headers, too.  Admitedly it does it in a
slightly weird way, adding only a Resent-To and relying on the MTA/MSP
to add the Resent-From.  But it does roughly the right thing, at least
when used with sendmail.

     -roy

---

From: Andrew Newton <[email-masked]>
Date: Mon, 19 Jul 2004 20:45:52 -0400

> the header was malformed.

Makes sense to me.

-andy

---

From: Mark Lentczner <[email-masked]>
Date: Mon, 19 Jul 2004 21:32:29 -0700

> anything substantially different from what the PRA algorithm
Well, when Caller-ID was first presented to me, it was presented as a 
heuristic.  And later evolutions of the algorithm seemed to get even 
more based on heuristics.  I was told that the X-Envelope, etc... 
things were added as they bettered some score on a test of "typical 
mail".

However, I agree with you that the current form attempts to encapsulate 
some notion of "who sent you this mail", as claimed by the headers.  
Alas, any such definition comes up against two problems: 1) RFC 2822 
doesn't talk in those terms, so we must infer, and 2) Current wide 
spread usage may or may not conform.

For example, RFC 2822 says "Resent fields are used to identify a 
message as having been reintroduced into the transport system by a 
user." -- and yet none of the mailing lists I use, which seem to be 
doing precisely this, add such headers.  Nor do any of my mailers when 
I choose "Forward".  So, practice and RFC are in conflict.  When, in 
practice, are these fields added?  Almost none of the mail I scanned 
had them.  On the other hand, wide spread use of a PRA check will 
probably cause wider use of the headers, in situations which may or may 
not be warranted.

On the other hand, RFC 2822 says "Resent fields are strictly 
informational.  They MUST NOT be used in the normal processing of 
replies or other such automatic actions on messages."  So, while it 
might represent who re-injected the mail, the user is to treat the mail 
as coming from the Sender:/From: addresses.  Hence, perhaps that is the 
identity that should be checked.  I have seen it requested that MUAs 
should change to show the PRA address to users as the sender of the 
e-mail, but that would seem to be in direct opposition to RFC 2822's 
intent of these headers.  Though I'm sure this is open to 
interpretation.

I point this out not because I have a strong feeling one way or the 
other, but because I think it should be clear that we are going to have 
to interpret 2822 and make some choices, and this will result in 
changes in practice.  Hence, I think it is reasonable to investigate, 
via both technical arguments and statistics, how well these ideas work.

> see anyone make a convincing argument for any other header identity.
See above for perhaps why just Sender:/From: could be argued for.

> implies using Sender in preference to From).
Well, to the end user this may not be so clear.  Because of the 
fractured nature of the mail identities (who gets bounces, who 
injected, who will get replies, who the user will see as the sender, 
etc...), there are several options for what you might want to check.  
The IP address isn't necessarily tied only to the concept of who 
injected it.  This is because the IP that injected the mail is in 
control of (can forge) any of those other identities.  So an IP-base 
authentication scheme is essentially asking a domain "Is this IP 
authorized to inject mail with your domain name in the following 
identity?".  I can see how a reasonable argument could be made for any 
of those identities.  As an end user I might even want them all 
checked, and flagged when I try to base actions upon them.

> analysing it semantically, not statistically.
If the semantics of mail identity were clear cut, and common practice 
generally compliant, I'd agree.  Alas, neither of these are true.  I 
think both semantics and statistics (testing with real world data) will 
have to come into play.

> contributors to this thread, but to the WG as a whole.)
No offense taken, and hopefully none given.

I also don't want others to take this as a position against PRA.  PRA 
is a fine identity to check.   As we come closer to some consensus on 
PRA, and as we fine tune it, I want express how I view this check as a 
guide to how we might refine it.

	- Mark

Mark Lentczner

---

From: Tony Finch <[email-masked]>
Date: Tue, 20 Jul 2004 11:23:29 +0100

On Mon, 19 Jul 2004, Mark Lentczner wrote:
>
> For example, RFC 2822 says "Resent fields are used to identify a message as
> having been reintroduced into the transport system by a user." -- and yet none
> of the mailing lists I use, which seem to be doing precisely this, add such
> headers.

I think the choice of the word "user" is intended to exclude mailing
lists.

> Nor do any of my mailers when I choose "Forward".

The Resent- headers are explicitly not for forwarding messages, in either
the encapsulated message sense or in the alias address sense. Read the
second paragraph after the syntax in section 3.6.6 of RFC 2822. This has
implications for other parts of the current MARID drafts, but PRA itself
is OK.

> So, practice and RFC are in conflict.  When, in practice, are these
> fields added?

Using the "bounce" (i.e. resend) function in Pine or Mutt, for example.

> On the other hand, RFC 2822 says "Resent fields are strictly informational.
> They MUST NOT be used in the normal processing of replies or other such
> automatic actions on messages."  So, while it might represent who re-injected
> the mail, the user is to treat the mail as coming from the Sender:/From:
> addresses.  Hence, perhaps that is the identity that should be checked.

That wouldn't make sense. An example use of the resend function where I
work is our staff who read webmaster or postmaster email resending a
message to the person best able to deal with it. Clearly this person
should reply to the original sender of the message since that is who has
the problem, but they received the message from the webmaster/postmaster
staff so that is what should be checked.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
CAPE WRATH TO RATTRAY HEAD INCLUDING ORKNEY: SOUTHEAST 3 INCREASES 5 TO 7.
SCATTERED SHOWERS THEN RAIN. GOOD BECOMES OCCASIONALLY MODERATE. SLIGHT BUILDS
MODERATE OR ROUGH IN EAST.

---

From: wayne <[email-masked]>
Date: Tue, 20 Jul 2004 10:47:14 -0500

On the same day that Caller-ID was announce to this mailing list
(April 6th, about three and a half months ago), I posted the following
question:

: In <[email-masked]> "Harry Katz" <[email-masked]> writes:
: 
: > [snip] But let's start with the base case - one address on the From line
: > - and work our way from there. 
: 
: I am very concerned with, as you say, the many special cases of
: dealing with the RFC2822 data.  While working code is not required at
: this stage of RFC development, it is certainly very helpful in trying
: to get a grasp on the whole problem and learning about subtle, but
: critical details.
: 
: Do you have any working code that validates the RFC2822 that we can
: look at, test, analyze and collect data with?  I realize that the C-ID
: doc on the microsoft website has an algorithm outline, but that would
: require a lot of work to turn into working code.
: 
: If you can't provide working code, can you provide data and your own
: analysis of the situation?

I would like to thank both Andy and Mark for finally providing some
data on the Caller-ID algorithm (aka PRA/PRD).  I think it is
especially nice of them since neither of them are the ones that have
been pushing for the PRA the hardest.

This working group is supposed to do engineering ("application of
scientific and mathematical principles to practical ends"), and it is
hard for me to believe that a huge amount of engineering has gone on
WRT the PRA when there has been no to base things on.

> ====   Msgs w/usable RR Deny  Pass

Andy:

I'm not sure what these numbers are supposed to be.  What exactly are
"messages with usable RRs"?  Are you saying that about one third of
all email that you checked had an SPF record for the domain?  If so,
that is higher than the data I'm seeing.

The "deny" percentages are also much higher than I would expect.  Have
you checked into why these messages are being rejected?  Are they due
to forgeries, or badly constructed SPF records, or fundamental flaws
in SPF-classic/SenderID?

In <[email-masked]> Mark Lentczner <[email-masked]> writes:

> domain of the envelope from.  [snip]  The
> [mongo snip!]

My guess is that the higher rate of 2821.FROM differing from the PRA
is caused by email sent through mailing lists.  I think it would be
interesting to dig into this to make sure we know what is going on.

> because the PRA is more often a bigger site.

Again, I suspect that this has to do with mailing lists.  From
skimming my own mail folders, it appears that domains that host
mailing lists are much less likely to publish SPF records.

> 4) For ham, env. from produces a greater percentage of passes than
> PRA.  It also produces few fails, but the numbers here are very low.
> For ham, more passes and fewer fails is good.

Yes, this is a very small test case, but both your data and Andy's
seems to show that the PRA has a higher rate of rejecting valid email
than SPF-classic.

I think more interesting numbers are actually in the attachment that
Mark included that had more detailed data.

> ::: Data set Ham 1 :::
> 
> PRA Query Stats:            published SPF
> ----------------          ---------------
>   pass                          11 ( 24%)
>   fail                           0 (  0%)
>   softfail                       0 (  0%)
>   neutral                       34 ( 75%)
>   unknown                        0 (  0%)
>   error                          0 (  0%)
> 
> Env. From  Query Stats:     published SPF
> -----------------------   ---------------
>   pass                          18 (100%)
>   fail                           0 (  0%)
>   softfail                       0 (  0%)
>   neutral                        0 (  0%)
>   unknown                        0 (  0%)
>   error                          0 (  0%)

Both Andy and Mark appeared to concentrate more on the pass/fail
results, but I think we need to look much closer at the neutrals.
There are quite a few major email domains that are currently
publishing SPF records that end in ?all.  I think we all want to see
the day when most people publish -all, so we need to look at why the
neutrals are happening and what can be done to make them pass.

My guess is, again, mailing lists.  If [email-masked] sends email
to [email-masked] and it is received by Andy or
Mark, the result will be a neutral for the PRA but for the 2821.FROM,
there will be no SPF record found.  This is because AOL publishes an
SPF record that ends in ?all, and Yahoo Groups doesn't add a Sender:
header to their mailing lists, although the do correctly use a Yahoo
domain for the 2821.FROM.

So, in this first set of ham, there are no outright rejections of
valid email for either PRA or SPF-classic, the PRA looks like it might
be generating a huge number of incorrect results.

> ::: Data set Ham 2 :::
> 
> PRA Query Stats:            published SPF
> ----------------          ---------------
>   pass                          14 ( 53%)
>   fail                           7 ( 26%)
>   softfail                       0 (  0%)
>   neutral                        3 ( 11%)
>   unknown                        2 (  7%)
>   error                          0 (  0%)
> 
> Env. From  Query Stats:     published SPF
> -----------------------   ---------------
>   pass                          16 ( 66%)
>   fail                           0 (  0%)
>   softfail                       0 (  0%)
>   neutral                        1 (  4%)
>   unknown                        7 ( 29%)
>   error                          0 (  0%)

Again, the PRA has returned more neutrals and, percentage wise, a huge
number of fails.

As both Andy and Mark pointed out, these are small datasets and are
probably not very diverse.  It is far too little data to draw any
conclusions from, but it does show that much more data needs to be
looked at.

-wayne

---

From: Margaret Olson <[email-masked]>
Date: Tue, 20 Jul 2004 12:36:39 -0400

> been pushing for the PRA the hardest.
The thing I don't understand about all this data is - what is the PRA 
analysis and particularly the PRA "pass/fail" analysis based on? 
Caller-ID records? It's hard to imagine that there is significant 
SenderID volume yet, since the specs are so new. There are not many 
Caller-ID records, let alone a representative cross section of domains.

Evaluating the PRA based on records published for SPF-classic is not 
valid. SPF-classic publication is itself a non-representative 
subsection of the net, and on top of that SPF classic records are not 
Sender ID records - the semantics are different - so "pass" and "fail" 
does not mean anything.

Also, where are the input mail samples coming from? If it is the 
inboxes of technical folks then it is not representative. To get a 
representative cross section of mail you need to look at a random cross 
section of consumer mailboxes on the big ISPs and also, separately, 
mail coming into corporate domains.  B to B and B to C mail have very 
different characteristics. You also have to make sure the inbox owners 
agree with your "ham/spam" assignments.

I agree that generating meaningful data here is extremely difficult, 
but how are these stats anything other than anecdotes with numbers? I 
agree it's very interesting, but I don't see how this data can be used 
for anything other than an aide to designing conclusive tests.

---

From: Andrew Newton <[email-masked]>
Date: Tue, 20 Jul 2004 13:39:34 -0400

> that is higher than the data I'm seeing.

That is what I'm saying.  And I would agree that it isn't 
representative of the domain space.  What it really points out is that 
I have a very non-diverse data set.

> to forgeries, or badly constructed SPF records, or fundamental flaws

It would be the latter since those were all ham and I had no dns errors 
or bad syntax in any of the ham (not true for the spam).  Also another 
indicator that my data set was not well diversified.

-andy

---

From: Greg Connor <[email-masked]>
Date: Mon, 26 Jul 2004 12:08:15 -0700

> Sender ID records - the semantics are different - so "pass" and "fail" 

Actually, this is exactly what SenderID proposal is based on -- the
assumption is that published SPF records will be effective when used
against the PRA.  The proposal depends on this, so it's good that it's
actually getting tested.

I see PRA as a first step of possibly several in our quest to determine
what identity to check.  Eventually, if Submitter becomes widely used,
we will reach a point where SUBMITTER is either there, or we fall back
to MAIL FROM, and then we will be close to where SPF Classic would be if
we had continued down the SRS path.

---

From: Michel Bouissou <[email-masked]>
Date: Mon, 26 Jul 2004 21:21:01 +0200

Le lundi 26 Juillet 2004 21:08, Greg Connor a écrit :
>
>  Eventually, if Submitter becomes widely used,

Is there actually a technical reason why we shouldn't continue down the 
Classic-SPF + SRS path ? SRS makes sense to me, much more that SUBMITTER and 
PRA which interest is yet very unclear in my mind (but the complication it 
introduces is ;-)

-- 
Michel Bouissou <[email-masked]> OpenPGP ID 0xDDE8AC6E

---

From: Douglas Otis <[email-masked]>
Date: Mon, 26 Jul 2004 12:47:24 -0700

> we had continued down the SRS path.

A typical technique used by spammers is to forge the RFC 2821 MAIL FROM
as a "back door" means to submit messages into a domain that would have
prohibited direct assess.  With focus moved from a useful check of the
RFC 2821 MAIL FROM to some RFC 2822 sender identity, how does this
ensure this "back door" remains closed?  The sender identity could point
to any random open SPF record or to one specifically designed to gain
full approval from a PRA evaluation.  PRA rather than Classic SPF
ensures the "back door" may never be closed.  Classic SPF can be made
much better by including BATV.

I would prefer to see the BATV be used in conjunction with the SPF
record.  If the SPF record were closed, then this could over-ride the
BATV signature.  The BATV signature would not require a full listing of
all domains, but could serve as a useful means to list just the domain's
outbound SMTP clients irrespective of other domains that carry their
mail.  This could become the preferred method, if a DomainKey technique
allows verification of the RFC 2821 MAIL FROM.

A 'replay' of this bounce address could be easily caught and keeps the
"back door" closed.  MAIL FROM addresses that do not use the BATV
signature and have a SPF record could then apply the matching address
requirement as a means of thwarting spoofed MAIL FROM addresses.

Looking at address records to sort mail based upon RFC 2822 identities
into different folders supports filtering and little else.  Filtering in
general will only lower the integrity of the mail system.  This seems to
be a bad direction to take, if there is a means to actually stop abuse
before it enters the mail stream.  A CSV accreditation by name used in
conjunction with strong checks on the RFC 2821 MAIL FROM have the
ability to close the door on abuse.  I would not expect SPF records to
be comprehensive enough in all cases, where a signature scheme could
take over in those cases as with BATV.  A signature method would also
provide a solution to many of the problems seen just using addresses
alone.

-Doug

---

From: Douglas Otis <[email-masked]>
Date: Mon, 26 Jul 2004 12:57:58 -0700

> PRA which interest is yet very unclear in my mind (but the complication it 

I agree.

I would hope this could be the direction taken.  I would see SRS being
transformed into incorporating BATV or perhaps the other way around. 
Either way, I see NO use for the PRA or Submitter scheme unless the goal
is to break mail and have messages go missing. ;^)

-Doug