Subject: RE: the focus of MARID

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Thu, 8 Jul 2004 07:24:28 -0700

I still fail to see how the identities matter.

All we are doing here is listing the IP addresses of our outgoing
mail servers.

I can be absolutely certain that is all we are doing since that is
all that we will be telling the network admins to do. 

The consequence of that is that by applying our knowledge of how the
infrastructure works we can make some deductions.

The SUBMITTER spec means that we can extend the scope of those 
deductions further to include RFC 2812.

The forwarding problem means that we can only ever validate the last
link in the chain and that as a result it is only ever possible
to know with certainty that a message is genuine, it is not possible
using the information from Sender-ID ALONE to know that a message is
defnitively fake if it purports to be forwarded.

I think that what we should be doing at this point is discussing
how to express the set of IP addresses. That is the part that has
the impact on what other people do.

The main complaint I and many others have of SPF is that it has
much too much flexibility. Even if you accept the need for
factored records, the macro language is far more powerful than
is necessary for that purpose.

	Phill

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of wayne
> Sent: Thursday, July 08, 2004 3:15 AM
> To: IETF MARID WG
> Subject: the focus of MARID
> 
> 
> 
> In <[email-masked]> "Alan DeKok" 
> <[email-masked]> writes:
> 
> >   My focus in ASRG & here in MARID has been RFC 2821 identities.  If
> > MARID is no longer discussing RFC 2821 identities, then 
> that's news to
> > me.
> 
> It is my understanding that, according to the MARID charter, the
> chairs would choose identities for this working group to focus on, and
> other identities would be ruled out of scope.  See the MARID
> description, paragraph four on:
> http://www.ietf.org/html.charters/marid-charter.html
> 
> The identities selected were the 2821 identities, not the 2822
> identities.  See:
> http://www.imc.org/ietf-mxcomp/mail-archive/msg01161.html
> 
> The Sender-ID proposal, for example, deals with the 2821 SUBMITTER
> identity, so it is in-scope.  (Or, at least that's the explanation
> Andy gave me a while back.)
> 
> 
> Now, I must admit that I am somewhat confused on this subject.  For
> example, I seem to recall several hums at the very end of the May
> interim meeting that changed the focus from 2821 to 2822, but they
> weren't in minutes and others didn't seem to remember such a thing.
> Who knows, I could easily be confused on this also.
> 
> 
> Speaking of confusing and minutes, I know that draft minutes for the
> May Interim meeting were posted to this list, but I don't remember
> seeing any final minutes and I can't find them on the MARID charter
> web page.
> 
> 
> Will minutes be published for the Interim meetings?
> 
> 
> -wayne
>

---

From: Greg Connor <[email-masked]>
Date: Thu, 08 Jul 2004 12:16:07 -0700

> all that we will be telling the network admins to do.

I agree with Phill here.

> defnitively fake if it purports to be forwarded.

Agreed.  The message should be shown as "from the forwarder" because that 
is the only step we are able to verify by IP.  (If the receiver is not 
using a forwarding service, he will probably see the actual sender's 
verification, in which case LMAP has done its job and all is well.)

In the future, if MARID is a success, I can see *maybe* applying a 
whitelist of trusted forwarders, and if the forwarder (or list robot) is 
trusted, using the previous-hop info that they recorded.  But, this is a 
long-term stretch goal.  MARID-brand LMAP has value in/of itself even if we 
never take this second step.

> is necessary for that purpose.

Yes, this is a frequent complaint, though I think each of the macros are 
there because someone came up for a reasonable explanation why they might 
be needed (maybe for logging or creating bounce-explanation messages and 
not for actual delivery).

Would you like to take a stab at naming some macros to be dropped?  (And 
yes, it's a serious suggestion; I'm not trying to be facetious or anything 
:)

--
Greg Connor <[email-masked]>

---

From: Meng Weng Wong <[email-masked]>
Date: Thu, 8 Jul 2004 16:38:46 -0400

On Thu, Jul 08, 2004 at 12:16:07PM -0700, Greg Connor wrote:
| 
| >The forwarding problem means that we can only ever validate the last
| >link in the chain and that as a result it is only ever possible
| >to know with certainty that a message is genuine, it is not possible
| >using the information from Sender-ID ALONE to know that a message is
| >defnitively fake if it purports to be forwarded.
| 
| Agreed.  The message should be shown as "from the forwarder" because that 
| is the only step we are able to verify by IP.  (If the receiver is not 
| using a forwarding service, he will probably see the actual sender's 
| verification, in which case LMAP has done its job and all is well.)

I would like to suggest we keep in mind:

  Where the mass majority of normal email is concerned,
  forwarding is an edge case.

  Failure of SenderID due to noncompliant forwarding is a
  corner case.

Therefore the failure scenario should be noted in detail and
announced to the industry so that troubleshooters can
recognize it when it happens.  But it is not big enough a
problem to keep us from moving forward.

We should also keep in mind the positive scenario for SenderID:

  If the mail is sent directly from Citibank to enduser,
  then SenderID returns an authentication PASS, and
  the reputation system returns a thumbs up,
  so the message gets a positive overall result,
  and the MUA displays a little green light or whatever.

The positive scenario is the scenario we are trying to enable.

At the highest levels, this is also true:

  we're moving toward a paradigm of "assumed guilty until
  proven innocent" (AGUPI).

In the AGUPI paradigm, the positive scenario becomes the
goal, and anything that falls short is deemed to be in the
YMMV zone.  Legitimate senders are assumed to have the
means, motive, and the opportunity to do what it takes to
get themselves into the positive scenario.  Legitimate
forwarders are also assumed to have the means, motive, and
opportunity to do their part, which is to say, prepend a
header and add SUBMITTER.

			   * * *

I also want to point out that SUBMITTER:
 - should appear rarely,
 - is more flexible than it seems, and
 - is amenable to AGUPI practices, which is to say
   unrecognized SUBMITTERs can be safely rejected.

I will explain each of these points.

* SUBMITTER should appear rarely.

SUBMITTER need not be used when the MAIL-FROM is identical
to the Sender.  This is a very common case.

* SUBMITTER is more flexible than it seems.

Even if the MAIL-FROM is different from the Sender,
SUBMITTER may still not need to be provided by the sender
MTA.  Why?

SenderID stipulates that the PRA must match the SUBMITTER; a
transaction in which PRA does not match SUBMITTER is deemed
to be noncompliant.

In discussion with Daniel Quinlan, he asked if that means an
MDA needs to have access to SUBMITTER.  This is when it
dawned on me that in practice, PRA doesn't always need to
match SUBMITTER.  After all, it's meant as an optimization.
If it isn't there, the PRA algorithm still works.

A receiver need test only two of the following three:

 - The PRA in the headers authenticates and passes reputation.
 - The SUBMITTER in the envelope authenticates and passes reputation.
 - The PRA must match the SUBMITTER.

This opens the door to a disconnect between PRA and
SUBMITTER.  Is this exploitable?  I don't think so, because
for anti-phishing purposes, we really only want the
end-user-visible address (the PRA) to authenticate and pass
reputation.  SUBMITTER is an optimization.  If the SUBMITTER
authenticates, and the PRA authenticates, then they're
really both okay.  They don't have to be the same!

Example:

  MAIL FROM:<[email-masked]> SUBMITTER=<[email-masked]>
  Resent-Sender: <[email-masked]>
  From: <[email-masked]>

The PRA is [email-masked].
SUBMITTER is [email-masked].

But both will pass authentication!

But anyway, this is a distraction.

* SUBMITTER can be safely rejected when it is not
  recognized.

When does SUBMITTER occur?  In forwarding.  What is the
difference between legitimate forwarding and spam trying to
masquerade as a forwarder?  In legitimate forwarding, I know
that [email-masked] forwards to me; it is my alias, and
I am responsible for setting up that relationship.  If I see
mail that says

  MAIL FROM:<[email-masked]> SUBMITTER=<[email-masked]>

I know that it's good.

If I see

  MAIL FROM:<[email-masked]> SUBMITTER=<[email-masked]>

then maybe it's a spammer trying to joe-job
[email-masked].

If the above is true, we can apply AGUPI.  We can simply
construct a per-recipient whitelist of recognized SUBMITTER
addresses.  And we can reject everything else.

So I hope the above discussion has illuminated the role of
SUBMITTER in more detail.

---

From: Dave Crocker <[email-masked]>
Date: Fri, 9 Jul 2004 15:07:19 +0700

Meng,

MWW> I would like to suggest we keep in mind:
MWW>   Where the mass majority of normal email is concerned,
MWW>   forwarding is an edge case.
MWW>   Failure of SenderID due to noncompliant forwarding is a
MWW>   corner case.

I would like to suggest that we keep in mind:

  We do not have statistics about the extent of use of forwarding
  scenarios, although we do know that it is used quite a lot, by many
  people and in a variety of ways.

  Changing a legitimate practise that has widespread use tends to be a
  good way to get the community to reject a proposal.

So, let me suggest that trying to marginalizing the problems caused
with forwarding scenarios is not such a good idea.

MWW> We should also keep in mind the positive scenario for SenderID:

Perhaps you are suggesting modifying the specification -- by the way,
are talking about marid-core? if not, then what specification are you
discussing? -- to explicitly list the scenarios for which it works
well, that is, the ones for which it is not problematic? That would be
very helpful.

MWW> The positive scenario is the scenario we are trying to enable.

Right.  Unfortunately, global standards usually need to work across
the entire architecture of the service they are modifying.

MWW> I also want to point out that SUBMITTER:
MWW>  - should appear rarely,

this is because you believe that rfc2821.mailfrom and
rfc2821.submitter will usually be the same?

MWW> * SUBMITTER should appear rarely.
MWW> SUBMITTER need not be used when the MAIL-FROM is identical
MWW> to the Sender.  This is a very common case.

So you do not mean that the syntactic form should appear rarely, not
that the semantics of submitter-based authentication and authorization
should get used rarely?

But the semantics are the issue, not the syntax.

d/
--
 Dave Crocker <mailto:[email-masked]>
 Brandenburg InternetWorking <http://www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>, <fax:+1.866.358.5301>

---

From: Tony Finch <[email-masked]>
Date: Fri, 9 Jul 2004 11:45:49 +0100

>   people and in a variety of ways.

We have about 32,000 accounts on the University of Cambridge's central
message store, and about 7400 (23%) of them have some kind of Sieve
redirect set up. 4700 of them are to non-Cambridge addresses, i.e.
FIFTEEN PERCENT of our users will have trouble with the deployment of
designated sender schemes such as SPF and SenderID.

Note that this is a view of outgoing email; it's much harder to tell what
proportion of our users have forwarding aliases that direct email here. If
the outgoing figure is any kind of indicator it would be irresponsible of
me to enable SPF checks on incoming email. I'll be disabling that part of
SpamAssassin 3.0 when I deploy it.

> We should also keep in mind the positive scenario for SenderID:

I think this is a bad idea, because it is the negative scenario that is
the problem. Neglecting that means you are not clearly defining the cases
in which your protocol can be reliably used to identify junk email.

>   proven innocent" (AGUPI).

I note that no satisfactory fixes have been suggested for the forwarding
problem. Some, such as SRS and the PRA algorithm require modification of
the email software at any site which implements forwarding, and in the
case of Cambridge it will require a complete re-engineering of our virtual
domain system with a significant increase in complexity and decrease in
performance.

Others, such as trusted-forwarder.org, drive a coach and horses through
the security model and completely negate AGUPI. If SenderID is ratified
and people start using it, I would expect a very large proportion of the
academic world to sign up to trusted-forwarder.org which would in turn
make them a really choice target for hacker-phishers.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
ST DAVIDS HEAD TO COLWYN BAY, INCLUDING ST GEORGES CHANNEL: NORTHWEST 4 OR 5,
OCCASIONALLY 6 BECOMES WEST TO NORTHWEST 3. RAIN OR SHOWERS BECOMES MAINLY
FAIR LATER. MODERATE OR GOOD. OCCASIONALLY ROUGH AT FIRST, OTHERWISE MODERATE
DECAYING LOCALLY SLIGHT.

---

From: Tony Finch <[email-masked]>
Date: Fri, 9 Jul 2004 14:15:45 +0100

> designated sender schemes such as SPF and SenderID.

I just realised that I counted everyone twice because of our replication
system, so these proportions should be more like 11% and 7%. The stats
from Oxford University are within a percentage point or so of my corrected
numbers, and more broadly in line with what I expected before running the
script. This is still a significant proportion of our user base.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
LUNDY FASTNET: WEST OR NORTHWEST 3 INCREASING 4 OR 5, OCCASIONALLY 6. SHOWERS
THEN RAIN. GOOD BECOMING MODERATE.

---

From: Tony Finch <[email-masked]>
Date: Fri, 9 Jul 2004 14:15:45 +0100

Tony.