Subject: RE: SPF abused by spammers

From: "Sauer, Damon" <[email-masked]>
Date: Thu, 9 Sep 2004 15:16:05 -0500

> > You are missing where this is a GOOD thing. We WANT spammers to use 
> > SPF. This will allow us to identify, publish, process, 
> shred, pillage, 
> > burn, destroy the IP addresses that this stuff is coming 
> from. Nobody 
> > is doing less stringent processing of the email that passes an SPF 
> > check. It just makes it easier to block when identified.
> 
> How do you know the spammer has not bothered to include addresses of 
> legitimate MTAs?  The obvious adaptive strategy would be to 
> disrupt this presumptive use of spammer's information as-if 
> it were trustworthy.  

 I am not looking at the SPF record at this point. I am looking at the
IP address they connected from.
 Am I misunderstanding your question? This just seems too obvious to me.

> 
> > The checks that I DON'T have to do are against the received from: 
> > headers. I already know. If you want to call this 'reducing 
> filtering' 
> > so be it. But it is a reduction because the check that you used to 
> > have to run is no longer necessary.
> 
> The value of SPF is clear when white-listing as a means for 
> reducing false negative assessments.
> 
> SPF will never serve as a tool for blacklisting for what 
> should be obvious reasons.  There is a risk presuming the 
> domain identified using SPF has not been spoofed somewhere in 
> the mail channel.  SPF does not allow the IP address to be 
> trusted to allow address blacklisting beyond the current 
> connection, nor does SPF really allow the MAIL FROM mailbox 
> domain be trusted to a degree that would allow name 
> blacklisting, as it fails to accurately identify the entity 
> introducing the message.  The mail channel is often shared 
> and there is no means to verify the channel is being checked 
> at either end of the administrative realms.
> 

 Please provide an example of how this would happen. I am completely
missing your chain of thought.

Regards,
Damon Sauer

*****
The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential, proprietary, and/or privileged material.  Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited.  If you received this in error, please contact the sender and delete the material from all computers. 113

---

From: Douglas Otis <[email-masked]>
Date: Thu, 09 Sep 2004 14:33:34 -0700

On Thu, 2004-09-09 at 13:16, Sauer, Damon wrote:
> > > SPF. This will allow us to identify, publish, process, shred,
> > > pillage, burn, destroy the IP addresses that this stuff is coming
> > > from. Nobody is doing less stringent processing of the email that
> > > passes an SPF check. It just makes it easier to block when
> > > identified.
> > 

If the mailbox domain has been safely blacklisted, then no confirmation
with respect to the IP address from the SPF records is needed.  Should
abusive mail be discovered, it would be unsafe to conclude the SPF
mailbox-domain/IP address confirmation is tenable for establishing a
name blacklist.  Only the IP address has received sufficient
authentication for such a listing, but then this has not improved upon
current IP address blacklisting.  

>  Am I misunderstanding your question? This just seems too obvious to me.

If you are only looking at the address, then there is no advantage
obtained using SPF records. 
 
> > > filtering' so be it. But it is a reduction because the check that
> > > you used to have to run is no longer necessary.

This would seem to imply you are checking the mailbox domains? 

> missing your chain of thought.

There are many cases where an MTA is shared.  This may be due to
transparent interception, virtual hosting, ports blocked by providers,
where many of these configurations ARE effective at curtailing abuse. 
If one of these MTAs is not checking against the SPF record, or the SPF
record is open, then there is no assurance a mailbox-domain appears on
account of the entity that published the SPF record.  There is no
indication a check has been made, nor that the MTA is being shared.  A
breach in the mail channel integrity can happen within either the
receiving or sending realm.  An SPF blacklisting fails to locate the
accountable entity.

Beyond white-listing, SPF may even be viewed as dangerous, when there
are expectations more can be obtained using this mechanism than is safe.

-Doug

---

From: "Alan DeKok" <[email-masked]>
Date: Fri, 10 Sep 2004 11:25:15 -0400

Douglas Otis <[email-masked]> wrote:
> with respect to the IP address from the SPF records is needed.

  RFC 2821 header checking against authentication records in DNS
enables recipients to safely blacklist mailbox domains.

  i.e.  100% of messages from a domain pass RFC 2821 DNS checks, and
are spam: the recipient may choose to blacklist the domain.

  In the absence of those checks, the recipient can still blacklist
the domain, but has no way of knowing for sure if the messages are
forged, and thus an attack by a malicious third party, who wishes to
disrupt communications between the alleged originator and the
recipient.

>  Should abusive mail be discovered, it would be unsafe to conclude
> the SPF mailbox-domain/IP address confirmation is tenable for
> establishing a name blacklist.

  Are there reasons why this is true?

>  Only the IP address has received sufficient authentication for such
> a listing, but then this has not improved upon current IP address
> blacklisting.

  You are stating that your position is that your position is correct.

> There are many cases where an MTA is shared.
...
> A breach in the mail channel integrity can happen within either the

  Shared services means shared accountability.  If the recipient is
unable to distinguish between a "good" party on a shared MTA and a
"bad" party, then by definition, both parties fall into the same
classification.

  Alan DeKok.

---

From: Douglas Otis <[email-masked]>
Date: Fri, 10 Sep 2004 12:52:56 -0700

> are spam: the recipient may choose to blacklist the domain.

I would restrict this to the EHLO domain.  Using the MAIL FROM is making
an assumption of the integrity of the mail channel providing the
message.  There is nothing to substantiate such an assumption and 
defending a blacklist is not a trivial matter.  There is naturally a
reasonably strong authentication of the IP address through transport
protocol interaction.  The use of names MUST be held to at least the
same level of authentication.  A name, unlike an IP address, may disable
mail and, effectively, how a business is identified!

Those running blacklisting services MUST be prepared to defend negative
name based assertions, especially with a name's broader impact.  A point
of view that the mail channel "should" be secure and publishing an SPF
record represents an implicit agreement the mail channel shall be
considered secure is not defensible.  "Tough luck, find a better
provider!" does not address a possibility the provider may not be at
fault.  There are at least two realms that must be secure.

> disrupt communications between the alleged originator and the

Those checks with respect to MAIL FROM and even the more extreme From
with Sender-ID, still assumes the mail channel is secure.  In the case
of "open" records, it is assumed the outbound mail channel is not being
shared.  Where the normal implementation may be to publish the "nominal"
sources of a mailbox domain and leave the record "open" as a solution
for the forwarding issues, allows messages to suffer "validation"
promotion moving through the mail channel.

>   Are there reasons why this is true?

In short, the mail channel can not be assumed to be secure and not
shared.

>   You are stating that your position is that your position is correct.

I was responding to Damon's statement that he was not looking at the SPF
record for making a decision.  If only the IP address is being used,
then this has not improved upon the current IP address blacklisting. 

The address space is becoming more dynamic, but there are still
black-holes implemented in routers by some providers that tend to
permanently disable address space due to bad policies of companies now
long out of business.  Reclaiming this space may take decades.  Moving
to a name based reputation system allows a long history to exist without
the overhead needed to track this exchange of IP addresses.

Trojan proxies are largely an outcome of the need to commandeer an
address that a blacklist attempts to make scarce.  While the number of
names of the good actors should reflect real uses, the number of bad
actors will explode should a name blacklist become effective.  This will
require taking advantage of the history that a name naturally provides. 
This may seem like a poor choice, as the IP address space is bounded and
the name space is not.

To deal with such a difficult problem may require:

a) Retain good actors long term
   (proved worthy of the magnetic domains expended)
b) Establish policy based affiliations
   (agrees to Opt-In only perhaps)
c) Limit exposure from new domains
   (cap messages perhaps)
d) Improve vetting of domain name applications 
   (industry sponsored clearing house perhaps)
  
> "bad" party, then by definition, both parties fall into the same

Sharing an MTA is a highly effective method used by ISPs to abate abuse.
Reserve painting this technique with such a broad brush.  This wide
brush never uncovers an MTA that may be compromised or poorly
administered.  Holding those unable to take immediate corrective action
accountable will cause a most unwelcome backlash.  The party that must
be held accountable is the entity administrating policies of the MTA. 
That party is only identified by an authenticated EHLO domain or the IP
address.

Having the MAIL FROM or From reference a name list of EHLO domains
allows a simple and safe method of publishing either the typical,
compatible, "nominal" record, or the atypical, breaks-everything, 
"restrictive" record for outbound MTA servers.  Making the authenticated
EHLO domain visible and used as a basis for establishing reputations,
affords consumer protections superior to that of the PRA algorithm. This
approach allows a policy that always restricts the From, as example.

Establishing a name for the MTA allows a "single" DNS lookup to further
establish the Mailbox-domain to MTA relationship in a non-exploitable
manner, and not the few hundred lookups required by SPF.  When SPF is
used to express the nominal outbound mail channel, this invites the very
exploitation SPF had promised to curtail!  

SPF is good for white-listing. For white-listing, I like the binary RR
specified by RFC3123 by P. Koch, "A DNS RR Type for Lists of Address
Prefixes", June 2001.  There is a bit of a mess with CIDR notation in
SPF.  It would be better in the long run to keep this information
concise and as part of the standard DNS reference library.

-Doug

---

From: "Alan DeKok" <[email-masked]>
Date: Sat, 11 Sep 2004 09:53:03 -0400

Douglas Otis <[email-masked]> wrote:
> message.

  Perhaps you didn't read the sentence you quoted.  I explicitly
stated that the message passed DNS checks, and there is therefore NO
need to make assumptions about the integrity of the mail channel
providing the message.  In the scenario I described, by *definition*,
the mail channel has been verified to have integrity, so far as the
RFC 2821 headers go.

  Please address that issue.  Repeating the claim that a blacklist is
based on "assumptions" when I explicitly described a scenario where it
could be based on verifiable information is... not nice.

>  There is naturally a reasonably strong authentication of the IP
> address through transport protocol interaction.

  Yes, you've said that many, many times.  I understand that's your
position.  I'm trying to get you to understand my position.

> with Sender-ID, still assumes the mail channel is secure.

  For Sender-Id, maybe.  For MAIL FROM checks, absolutely not.

  Security != authentication.  You can authenticate someone in an
insecure system.

  MAIL FROM checks work in an insecure mail channel.  They *don't*
interact well with ".forward" systems, but that's an authentication
issue, not a security issue.

  In fact, MAIL FROM checks use the source IP address (which you claim
is the only secure/verifiable entity) to answer the question "is this
IP permitted to use this domain name in MAIL FROM?"  If we put
additional conflicts with ".forward" systems aside for a moment, then
MAIL FROM and EHLO based checks are identical in security.

> Reserve painting this technique with such a broad brush.

  I'm not.  I'm talking about what information is out there, and what
decisions can possibly be made, independent of any technical proposal.

  If someone can't tell the difference between two things, then it
MUST treat the two things as identical.  This is a well-known concept,
and based solely on the available information.  If you choose to
disagree with this, that's your perogative, but it means that any
system you come up with will be based on using information which is
unavailable to anyone.

> Holding those unable to take immediate corrective action accountable
> will cause a most unwelcome backlash.

  This is a political response to a technical problem: the recipients
cannot technically distinguish between two parties, and therefore
treats both the same.  If one party doesn't like it, the solution
isn't political, it's technical: give the recipient additional
information which lets him distinguish the two parties.

> The party that must be held accountable is the entity administrating
> policies of the MTA.  That party is only identified by an
> authenticated EHLO domain or the IP address.

  And other parties using that MTA will, by definition, get associated
with the reputation of that MTA.  This is what you call "a broad brush".

  Alan DeKok.

---

From: "Douglas Otis" <[email-masked]>
Date: Sat, 11 Sep 2004 11:47:59 -0700 (PDT)

> "Alan DeKok" <[email-masked]> wrote:
> providing the message.

A DNS check that obtains a list of addresses for a mailbox-domain, MUST
assume channel integrity when holding the apparent originator of the
message accountable.  A mailbox-domain administrator may delegate mail
services to many disparate sending domains which may also be shared with
many other mailbox-domains.  Publishing an SPF record does not include an
implied acceptance of accountability for these other sending domains or
the receiving domains within the recipient's realm.

Due to forwarding issues, expect many lists to be left "open" to allow
other unknown domains to forward messages.  It is unfortunate, with the
SPF draft, the expressed MTA/mailbox-domain relationship is used to also
authorize the sending MTA.  This means an "open" relationship will likely
invite a problem the SPF record hoped to prevent.  Complex macros aimed at
rediscovering the actual sending MTA or, in the case of Sender-ID, the PRA
algorithm used to indirectly resolve the sending MTA where the RFC2822
From had been superceded, can simply be replaced by authenticating the MTA
EHLO.

There are clearly two steps needed, but these two steps have been
collapsed into a one very broken step with SPF or Sender-ID.

Step 1: Authenticate the MTA EHLO name.
Step 2: Compare the mailbox-domain/EHLO list to this name.

If spam is detected at the MTA, the EHLO name is held accountable.
By having this second step, confusion regarding which entity must be held
accountable is removed and safely allows any level of restriction to be
placed upon either the MAIL FROM or the From mailbox domains.  By making
the authenticated EHLO domain visible within the MUA and allowing a few
institutions to publish a comprehensive rather than nominal
mailbox-domain/EHLO list as an exceptional restriction, will provide
superior consumer protection than possible with either SPF or Sender-ID.

> In the scenario I described, by *definition*, the mail channel has been
> verified to have integrity, so far as the RFC 2821 headers go.

It is not a trivial matter defending a reputation service.  Listing a name
has potentially greater impact than an IP address.  One can not defend
assumptions of accountable entities by suggesting the mail channel is
"defined" to have integrity.  Nor is it reasonable for those seeking to
have their mail accepted to be held accountable for the actions of others.
 SPF with "open" lists is seriously flawed, to say nothing about other
security issues.  Anyone sharing a domain and publishing an "open" list is
susceptible to being spoofed.  Because of the collapsing of the MTA
authorization with the domain relationships and allowances for "open"
lists, this actually invites spoofing.  While listing "nominal" outbound
servers may be useful for "coloring" the mail, these mailbox domains MUST
NOT be held accountable for any abuse that may occur.

> could be based on verifiable information is... not nice.

Listing a name and blocking their mail because you don't care which entity
actually caused the message to be sent is also... not nice.

> issue, not a security issue.

What entity should be held accountable?  The entity authenticated as
sending the mail, or the mailbox-domain that authorized the mail to be
sent?  Obtaining an authorization is not the same as authentication of the
acting entity.

> MAIL FROM and EHLO based checks are identical in security.

These checks are not the same.  You are devising a system that requires
mailbox-domains authorize other domains to send their mail.  Those
actually sending the mail can be authenticated as performing this
directly.  Simply authorizing a domain does not imply they accept
accountability for these sending domains.  If the sending MTA EHLO name is
authenticated and authorized by the EHLO domain, then it does not matter
what mailbox-domain is sent, this EHLO entity can be safely held
accountable for the MTA actions.  If there is a problem, only this entity
can take immediate action or provide logs in the event of a crime.

> MUST treat the two things as identical.

But these things can be identified by an authenticated EHLO name.  They
MUST NOT be treated as identical.

> This is a well-known concept, and based solely on the available
> information.  If you choose to disagree with this, that's your perogative,
> but it means that any system you come up with will be based on using
> information which is unavailable to anyone.

SPF is proposing to add information, but then uses this information
inappropreiately.  CSV and MPR attempt to add information and use this
information to establish accurate accountability.  There is a difference
and anyone can use this information.

> information which lets him distinguish the two parties.

That is what CSV does.  It allows a reputation service to distinguish
those directly responsible without any assumptions of the mail channel
integrity.

> with the reputation of that MTA.  This is what you call "a broad brush".

A reputation service spends most of their efforts ensuring information. 
Using the IP address makes this process rather straight forward.  Using an
authenticated EHLO name follows the same model and directly identifies
those sending the messages.  To suggest that because the EHLO domain was
not considered, means a reputation service can safely hold entities
accountable for the action of others will not have much sway.  Again, this
is not a trivial matter.

-Doug

---

From: "Alan DeKok" <[email-masked]>
Date: Sat, 11 Sep 2004 17:15:48 -0400

"Douglas Otis" <[email-masked]> wrote:
> message accountable.

  What channel?  The only parties involved are:

  a) SMTP client
  b) SMTP server
  c) DNS server being queried

  The DNS server may return one, or a billion IP's "permitted" to send
MAIL FROM with it's name.  Any particular message may have traversed
zero, or a billion intermediate SMTP hops before arriving at the
current SMTP client, and being sent to the SMTP server.

  I fail to see how any party other than the three listed above could
possibly be involved in "MAIL FROM" DNS checks.  There is no "channel"
beyond the current hop, which can be verified to exist.

> other unknown domains to forward messages.

  As has been noted here, as elsewhere, users affected by forwarding
issues are a very, very small percentage of people on the net.  As a
result, the reasonable expectation is that very few domains will have
"open" or permissive records.

> Step 2: Compare the mailbox-domain/EHLO list to this name.

  I'm not sure what you mean by the second step.

> One can not defend assumptions of accountable entities by suggesting
> the mail channel is "defined" to have integrity.

  You're not addressing my scenario, or my reasons for my position.
You're just labelling my position as based on "assumptions", and
therefore any "definition" I come up with is, in your opinion, defined
to be indefensible.

> actually caused the message to be sent is also... not nice.

  Which is why I didn't say that.  Please don't accuse me of saying "I
don't care", it's rude and insulting.  My position, if you had read my
messages, is that IF the recipient cannot tell the difference between
"good" and "bad" originators, then he MUST treat them as identical.
Because, by definition, he has no information by which to distinguish
them.

  Unless.. do you have a way for the recipient to treat "good" and
"bad" originaters differently, in the absence of any data which can be
used to distinguish them?  If you do, I would be... surprised.

> sent?

  Each has a different role, therefore their accountability should be
different.

>  Obtaining an authorization is not the same as authentication of the
> acting entity.

  I'm not sure how that applies to the issues on this list.

  An SMTP server can obtain authentication information from a DNS
server for an SMTP client, and use that information to authorize the
client.  If there's no authentication information available, then the
SMTP server cannot use that information to authorize the user.

> These checks are not the same.

  Which is why I didn't say they were the same.

>  You are devising a system that requires mailbox-domains authorize
> other domains to send their mail.

  No.  "domains" don't send mail.  SMTP clients send mail, and use
domain names in fields in SMTP.  The difference is crucial.  SMTP
clients can be correlated to domain names via:

  - rDNS
  - EHLO
  - MAIL FROM

  And not much else.  Because there is no easily available mapping of
"domain" to SMTP client, other than those fields, SMTP servers cannot
use anything other than those fields to tie the SMTP client to a
domain name.  So talking about "authorizing domains" is a
misconception.  SMTP clients are authorized, nothing else can be.

> If the sending MTA EHLO name is authenticated and authorized by the
> EHLO domain, then it does not matter what mailbox-domain is sent,
> this EHLO entity can be safely held accountable for the MTA actions.

  How?  In contrast, by checking MAIL FROM, you can at least have some
confidence that a bounce path exists, and any errant message will be
accepted by the originator.

> MUST NOT be treated as identical.

  If two domains use the same MTA, which uses only one authenticate
EHLO name, then by your definition, only the domain in EHLO can be
held accountable, and all recipients must therefore treat the two
domains identically.

  Hmm... have I used your definitions to prove my point?

> those sending the messages.

  It authenticates the MTA sending the messages, which is a little
different.

>  To suggest that because the EHLO domain was not considered, means a
> reputation service can safely hold entities accountable for the
> action of others will not have much sway.

  It's not unreasonable to hold multiple parties accountable for a
message.  ISP's are already holding other ISP's responsible for the
actions of their customers.  It's not unreasonable to use MAIL FROM
accountability in addition to EHLO.

  Alan DeKok.

---

From: Dean Anderson <[email-masked]>
Date: Sat, 11 Sep 2004 18:41:33 -0400 (EDT)

>   c) DNS server being queried

I think he is refering to the channel referenced by information theory
(guessing by the previous reference to Lampson's covert channel paper).
This could refer to the complete path through the communication system, or
it could refer to a single hop in a multihop system.

Further, you've oversimplified the parties involved:

	The end user, 
	optionally a third party relay [one or more]
		(because connection and mail services may be unbundled)
	The mailbox provider (where responses will go)
	The Mailbox provider nameservers
	The recipient mail server [one or more]
	The recipient nameservers
	The chain of nameservers between the recipient, root, and mailbox 
provider.

At any point spoofing could occur.

Going back to my spam examination where you are examining a spam message:
Suppose the IP address in the received from header no longer matches the
SPF records:  In this case you don't know if your SPF check passed because
DNS records were spoofed, or if the real DNS records were changed.  All 
you know is that you got a spam from a certain IP address.  You don't know 
whether it was forged or not.

Suppose the IP address in the recieved header does match the SPF records. 
You don't know if the abuser signed up for an account with that domain, or 
if they got an infected machine under that domain.  You don't know whether 
the nameserver was cracked, and incorrect records added. Most people can't 
tell the difference between a poisoned DNS cache and whether the records 
really come from an authoritative nameserver or even figure out where they 
got the DNS record from.   So, similarly, all you really know is that you 
got a spam from a certain IP address.

In both cases, all that can be done is report the incident to the operator 
of the domain and the operator of the IP address.  And that's all that can 
be done now. THERE IS NO CHANGE.

SPF doesn't prevent email forgery, nor even indicate whether forgery
happened.

> beyond the current hop, which can be verified to exist.

The current hop cannot be "verified to exist" beyond the TCP sequence 
numbers. There is no way to tell that the mail isn't forged.  Even if 
email comes from a certain server, there is still no way to tell that the 
mail isn't forged.  <user>@aol.com coming from an AOL mailserver doesn't 
mean that it really came from <user> Spammers can get accounts at AOL. 
Spammers can steal accounts at AOL by a number of vectors such as viruses.

And what's more, from outside of AOL we can't be sure that AOL isn't
pink-contracting with spammers.

The notion of "spam-herding" is just so much wishful thinking.

---

From: "Douglas Otis" <[email-masked]>
Date: Sat, 11 Sep 2004 23:02:47 -0700 (PDT)

>  c) DNS server being queried

The MAIL FROM mailbox-domain was passed through the mail channel, for
messages not originating at the SMTP client.  This is normally the case
for the SMTP client performing public delivery.  This adds at least an
additional SMTP client/server pair.

> beyond the current hop, which can be verified to exist.

For accountability based upon the mailbox-domain, each hop is involved
passing this identity through the mail channel.  Holding the administrator
of the mailbox-domain accountable would be ignoring the complexity of the
mail channel, and the potential change of validation status moving from
hop to hop.  Publishing an SPF record is not tantamount to a contract
accepting accountability for all published addresses.

> "open" or permissive records.

While it may be a small percentage of users forward mail or use a type of
list server, this generalization must not be applied to the mailbox
domain, as there are typically many users per domain.  SPF records are not
typically user based, but are domain based.  There will be expensive
support issues caused by closing these records.

>   I'm not sure what you mean by the second step.

First, authenticate the name of the MTA.  Second, check against list of
MTA names referenced by the mailbox-domain for a match.  See the marid-mpr
draft for more details.  This list is obtained using a single DNS lookup,
unlike SPF which may require dozens or hundreds of sequential lookups. 
This list can safely be declared "nominal" without being used as an
authorization exploit, as with SPF.  Depending upon the nature of the
list, comprehensive or nominal, the message can be rejected at the MTA or
"colored" at the MUA respectively.  If the message was abusive, the MTA
and NOT the mailbox-domain MUST be held accountable.

> therefore any "definition" I come up with is, in your opinion, defined

I am not attempting to dodge your scenario.  I do not agree you have
adequately described the situation or considered risks involved making
negative assessments against a name.

>> Listing a name and blocking their mail because you don't care which
>> entity actually caused the message to be sent is also... not nice.
>

I should have said "one" does not care.  Sorry for my phrasing.  I did not
intend to be rude.  I would hope the goal of the MARID group would be to
provide a method that accurately distinguishes those accountable for
abusive actions.  Currently the group seems intent upon skipping a vital
step.

> used to distinguish them?  If you do, I would be... surprised.

Except for the case where the mailbox-domain referenced a comprehensive
list and could not match the MTA name for authorization, no other
assertions are safe.  The concept of good and bad should be reserved for
reputations of the MTA name.

>   Each has a different role, therefore their accountability should be

Yes, but with respect to a reputation service, only the MTA name may be
safely assessed.

> client.

The client MUST be authenticated and the authorization validated when
holding the MTA name accountable and asserting a reputation for the MTA
name.  A message (or message action) authorization is not sufficient to
hold the mailbox-domain accountable and asserting a reputation for the
mailbox domain.

>  If there's no authentication information available, then the

The alternative becomes an IP address blacklist.

>   Which is why I didn't say they were the same.

You seem to suggest the MAIL FROM and EHLO identities are equally strong. 
They are not.

> clients can be correlated to domain names via:

I don't wish to get bogged down in semantics and protocol details. The
MAIL FROM name may not correlate to the to entity accountable for the
actions of the MTA.

> misconception.  SMTP clients are authorized, nothing else can be.

Forgive my generalizations that failed to include the protocol.

>   How?

The name of the MTA is held accountable.  A name, by any other name, is
still a name. :^)

> In contrast, by checking MAIL FROM, you can at least have some

This is only true for the comprehensive list.  But this does not mean it
is safe to hold this entity accountable for the messages being sent by the
MTA.  If anything, the MTA entity (EHLO name) should come into question.

>   Hmm... have I used your definitions to prove my point?

You missed the concept of what entity is safely accountable for messages
sent.  It can not be determined by the mailbox-domain.  When the name
affected by reputation services is the provider, then either this MTA is
rehabilitated, or their customers move to a different provider.  The
reputation service has not harmed the wrong party.

>> Using the IP address makes this process rather straight forward.  Using
>> an authenticated EHLO name follows the same model and directly identifies

Again, I was not including the protocol while explaining the entities
involved.

> accountability in addition to EHLO.

Reasonable will be determined by those unfairly harmed by overly broad
concepts of what a DNS record suggests.  Likely there will also be a fair
amount of painful litigation in the case of SPF or Sender-ID.

Perhaps there should be a warning in the SPF draft:

Publishing SPF records may result in a negative reputation status through
events beyond the control of the administrator.  Publishing "open" records
at your peril.

These and many other problems do not exist for CSV and MPR.  The other
potential effort. :^)

-Doug

---

From: "Alan DeKok" <[email-masked]>
Date: Sun, 12 Sep 2004 09:18:55 -0400

"Douglas Otis" <[email-masked]> wrote:
> The MAIL FROM mailbox-domain was passed through the mail channel

  WHAT "mail channel"?  I have described what I think of as the "mail
channel", and given reasons for my conclusions.  You haven't.

> for messages not originating at the SMTP client.

  I explained why the SMTP server cannot verifiably know that, and why
(for all practical purposes), the SMTP client is the only verifiable
originator of the message.  You haven't explained why you disagree, or
what errors (if any) exist in my message.

  Simply stating a contrary position is not an argument.

> This is normally the case for the SMTP client performing public
> delivery.  This adds at least an additional SMTP client/server pair.

  Which is effectively what I said in the paragraph below the one you
quoted, and where I explained why it didn't matter to the SMTP server.

> negative assessments against a name.

  The risks involved are undertaken by the mailbox-domain, when it
publishes MTA authentication records in DNS.  The scope of those risks
are defined by the documents under discussion in this group.  As for
my consideration of those risks, I've explained my reasons, and I
don't think I've seen you refute any of them.  You've just stated that
you disagreed, and then talked about your position.  That's nice, but
it doesn't give me any insight as to why you disagree with my
position, or what's wrong with my position.

> The concept of good and bad should be reserved for reputations of
> the MTA name.

  Why?  I've explained why and how MAIL FROM accountability can be
safely applied.

> A message (or message action) authorization is not sufficient to

  Why?  If any MAIL FROM authentication system is intended to hold a
domain accountable, then MAIL FROM authorization is perfectly valid
for holding that domain accountable.

> You seem to suggest the MAIL FROM and EHLO identities are equally strong. 

  Why?  And I'm not suggesting they're equally "strong", but that both
may be used, each for different purposes.  Saying "they have different
kinds of accountability" is a clear statement that one may be more, or
less, accountable than the other.

> I don't wish to get bogged down in semantics and protocol details.

  Uh... what else is there?  If we're not going to discuss the
protocols involved, then we're talking about "angels on the head of a
pin", and not doing engineering.

> The MAIL FROM name may not correlate to the to entity accountable
> for the actions of the MTA.

  And sometimes it may.  The two statements are not contradictory.
The MAIL FROM name may also correlate with the entity accepting
accountability for the *message*, which is what I've been saying all
along.  And that statement does not contradict, or invalidate, the
idea that an entity (possibly different0 is responsible for the
actions of the MTA.

  Alan DeKok.

---

From: "Douglas Otis" <[email-masked]>
Date: Sun, 12 Sep 2004 09:47:01 -0700 (PDT)

"Alan DeKok" <[email-masked]>
> channel", and given reasons for my conclusions.  You haven't.

Mail is normally passed from MTA to MTA on its journey to the destination.
The MTA making the public delivery is often not where the message was
created.  The sequence of Mail Transfer Agents carrying the MAIL FROM
identifier is what I had referred to as the mail channel.

> what errors (if any) exist in my message.

It is how the SMTP client is identified.  This identifier, to allow safe
reputation assertions, must reflect the identity of those administratively
in control of the sending machine, and able to take immediate corrective
action.  An SPF record does not give a reputation service license to hold
those that may have "authorized" the MTA to be accountable for the actions
of this potentially different entity.  Jim Lyon even stated an expectation
of holding the mailbox-domain accountable even when the MTA had not been
specifically "authorized" as a result of an "open" list.

>   Simply stating a contrary position is not an argument.

You are saying one can not be sure of the origin of the MAIL FROM, but it
is okay to use this identity to assert a reputation assessment against the
actions of the SMTP client.  I agree one can not be sure of the origin of
the MAIL FROM and, as a result, the mailbox-domain MUST NOT be used to
assert a reputation assessment against the actions of the SMTP client. 
There is an identity that accurately identifies the SMTP client.  This is
either the IP address or an authenticated EHLO.

> publishes MTA authentication records in DNS.

What assurance can you provide that supports this opinion?  There is AOL
asking people to publish SPF records so they may be white-listed.  Now you
are saying publishers should also expect to be black-listed when they
publish an "open" record as have AOL.  Is this do as I say and not as I
do?

> The scope of those risks are defined by the documents under discussion in
> this group.  As for my consideration of those risks, I've explained my
> reasons, and I don't think I've seen you refute any of them.  You've just
> stated that you disagreed, and then talked about your position.  That's
> nice, but it doesn't give me any insight as to why you disagree with my

Rather than justifying not obtaining the SMTP client name out of
expediency, consider the outcomes when reputation is used to block the
names obtained.  Showing the name being blocked is the entity that
actually sent the abusive mail will greatly aid the defense of the
reputation service.  To suggest it does not matter that the name being
blocked only authorized the SMTP client and therefore accepted
accountability for these disparate entities will be a much more difficult
position to take.  I would expect this position will not provide much
protection in what will likely be a sizable litigation.

>   Why?  I've explained why and how MAIL FROM accountability can be

You have not.  You simply say you can hold the MAIL FROM entity
accountable because they published with the clear warning within some
discussion within this WG.

> for holding that domain accountable.

It is not authenticating, it is authorizing the delivery of the message. 
The EHLO name can be authenticated, the MAIL FROM provides authorization
as does the CSV record in a generic sense.

>> You seem to suggest the MAIL FROM and EHLO identities are equally
>> strong. They are not.
>

The high risk act of asserting reputation demands authentication of those
entities directly accountable for the actions of the SMTP client, and not
based upon implied acceptance of accountability and assumed integrity of
the mail channel.

> along.

Yes you have.  But where does it say publishing an SPF record is accepting
accountability for any message using this mailbox domain coming from an
SMTP client?  That is the million dollar question.

> And that statement does not contradict, or invalidate, the
> idea that an entity (possibly different is responsible for the
> actions of the MTA.

I am of the opinion these assumptions are caviler and needlessly risky.
This risk will represent the demise of the reputation service.

-Doug

---

From: Dean Anderson <[email-masked]>
Date: Sun, 12 Sep 2004 18:26:00 -0400 (EDT)

> are spam: the recipient may choose to blacklist the domain.

Many years ago, people started to block domains and email addresses. It
was found that rather than blocking spam, this practice motivated more DOS
attacks (joe-jobs).  It would never be the case that 100% of the messages 
from a domain 'pass checks' and are spam, unless that domain has been 
subjected to a joe-job or the domain is disposable. If it is disposable, 
then blocking it won't have any effect, since it won't be used for long.

> disrupt communications between the alleged originator and the

One still has no method of determining if the messages are forged, checks 
or no checks. The "checks" have no value. The message could still be 
forged, the DNS records could be forged, the senders machine could be 
compromised, the abuser could obtain an account at the ISP that hosts the 
domain, etc, etc, etc.

		--Dean

---

From: "Alan DeKok" <[email-masked]>
Date: Sun, 12 Sep 2004 21:23:19 -0400

> Mail is normally passed from MTA to MTA on its journey to the destination.

  How does that affect MAIL FROM authentication via records in DNS?

> identifier is what I had referred to as the mail channel.

  None of which is relevant to MAIL FROM authentication via records in
DNS.  The previous hops are invisible to MAIL FROM checks.

> actions of the SMTP client.

  I'm saying that the recipient may not know the "true" origin, and
that that "true" origin may be un-knowable.  The recipient CAN know
that the current hop is authenticated, if the MAIL FROM satisfies
authentication records in DNS.  For the recipient, nothing else is
verifiable, and therefore nothing else is relevant.

  All I'm doing here is taking your EHLO argument, and applying it to
MAIL FROM.  Your opposition to this process indicates conflict or
confusion somewhere in the conversation.

> Now you are saying publishers should also expect to be black-listed
> when they publish an "open" record as have AOL.

  Publishers should expect recipients to do the most curious things,
for the most inane reasons.

> Showing the name being blocked is the entity that actually sent the
> abusive mail will greatly aid the defense of the reputation service.

  Replace "sent" with "accepted responsibility for", and MAIL FROM
checking is perfectly valid, if all parties agree.

> discussion within this WG.

  No... that's dumb.  "MTA's don't read mailing lists".

  In any protocol implementation, all parties involved agree as to
what the information means, and how to interpret it.  MARID
implementations are nothing special in this regard.  If this WG
publishes standards defining accountability, then it's entirely
reasonable to expect that implementors of the standard have accepted
that accountability.

> But where does it say publishing an SPF record is accepting
> accountability for any message using this mailbox domain coming from
> an SMTP client?  That is the million dollar question.

  That spec isn't under consideration by this group, so any answer
would be out of scope.  But in general, I believe close examination of
email scenarios may lead to such statements.

> This risk will represent the demise of the reputation service.

  Which is why I have great skepticism that any reputation system will
work.

  Having my ability to run an MTA depend on kissing up to some third
party?  Sure... that fits into the freedom, and the "end to end"
nature of the net.

  Alan DeKok.

---

From: Dave Crocker <[email-masked]>
Date: Sun, 12 Sep 2004 21:48:37 -0700

Alan,

>> Mail is normally passed from MTA to MTA on its journey to the destination.
AD>   How does that affect MAIL FROM authentication via records in DNS?

That depends upon the authentication scheme and the nature of the
information coming from the DNS.

However the fact that mail transits multiple MTAs is hardly remarkable
insight. It has been the nature of networked email for more than 25
years, so it's not clear what prompts your particular question.

>> identifier is what I had referred to as the mail channel.
AD>   None of which is relevant to MAIL FROM authentication via records in
AD> DNS.  The previous hops are invisible to MAIL FROM checks.

Actually, the fact that a mailfrom authentication is being performed an
arbitrary number of MTA hops away from the mailfrom's creation is
extremely relevant to the technical basis and administrative complexity
of maintain DNS information needed to achieve that authentication.

>> actions of the SMTP client.
AD>   I'm saying that the recipient may not know the "true" origin, and
AD> that that "true" origin may be un-knowable.  The recipient CAN know
AD> that the current hop is authenticated, if the MAIL FROM satisfies
AD> authentication records in DNS.

Correct. If all the information lines up correctly, you know something
useful.  If, however, the information is missing or does not line up,
then it is not at all clear you know anything.

And if the scenarios for which the information lines up turn out to be
few, then the entire mechanism has very limited utility.

Given the aggregate cost of Internet standards, it is useful to worry a
great deal about the technical, administrative, and operational
complexity of a proposal. I pretentiously call this it's "physics", in
the hope that folks will worry a bit about system dynamics. There is a
tendency to focus only on very localized aspects of a design.

d/
--
 Dave Crocker <dcrocker-at-brandenburg-dot-com>
 Brandenburg InternetWorking <www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>

---

From: "Alan DeKok" <[email-masked]>
Date: Mon, 13 Sep 2004 10:44:47 -0400

> years, so it's not clear what prompts your particular question.

  My focus in this thread has been MAIL FROM authentication via
records in DNS.  Could you explain how multi-hop transit affects the
current scope of an SMTP server performing such authentication?

  a) one of the hops fails to forward the message, in which case the
     current hop doesn't exist.

  b) all of the hops forward the message, in which case none of that
     information is visible in MAIL FROM.

  e.g. an SMTP server sees "MAIL FROM: [email-masked]".

  I have no idea how that server could use that data to determine that
any previous hops existed.  Therefore, for the purposes of MAIL FROM
authentication via records in DNS, those previous hops do not exist.

  Can you, or Doug explain how the previous hops are relevant to MAIL
FROM authentication via records in DNS?  While I understand Doug is
trying, all I've gleaned from his responses have been that "mail is
multi-hop".  This may be an attempt to say "a message must pass all
MAIL FROM checks in all hops", but I haven't seen that stated
anywhere, as it seems fairly obvious.

> of maintain DNS information needed to achieve that authentication.

  I agree, but that's an issue for the sender, not for the recipient.

  There are multiple scopes involved.  Each party has it's own scope;
it's own narrow vision of the net.  So far in this thread, I have
focussed on the scope of the recipient.  In that scope, none of the
previous hops exist, because they cannot be verified to exist.

> then it is not at all clear you know anything.

  I'm not sure what "line up correctly" means.  Most of these checks
have only negative value.  A "pass" gives you little information.  A
"fail" gives you more.

> the hope that folks will worry a bit about system dynamics.

http://adsabs.harvard.edu/cgi-bin/nph-bib_query?2000NIMPA.449..172B

  Nuclear Instruments and Methods in Physics Research Section A,
Volume 449, Issue 1-2, p. 172-207.

  I am not unfamiliar with such concepts.  My training is in
modelling, simulation, and development of large systems.  The methods
learned in that training are directly applicable to most systems,
though I'm having difficulty applying them here.

> There is a tendency to focus only on very localized aspects of a
> design.

  Because they are simpler than trying to understand the system as a
whole.  And if we don't understand the local aspects of a design, we
have very little hope of understanding the system.

  Alan DeKok.

---

From: Tony Finch <[email-masked]>
Date: Mon, 13 Sep 2004 16:34:56 +0100

> anywhere, as it seems fairly obvious.

What Doug means is that email can legitimately come to my servers from
an IP address that is not anticipated by example.com because there was an
extra hop in its journey: [email-masked] sent the message to
[email-masked] which forwarded it to
[email-masked]. Therefore example.com can't correctly say
that my servers will only receive email with an example.com bounce address
from their outgoing email servers, because as you say I have no way of
knowing about the message's previous hops, and example.com has no way of
knowing about recipients' forwarding arrangements.

I cannot work around this problem: I have no reasonable way of knowing
that this forwarding relationship exists, or if I do know about it I have
no reasonable way of maintaining a list of example.edu's outgoing email
servers in order to whitelist them.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
NORTH FITZROY SOLE LUNDY FASTNET IRISH SEA: WEST OR SOUTHWEST 6 TO GALE 8,
PERHAPS SEVERE GALE 9, VEERING NORTHWEST 5 TO 7. SHOWERS. MAINLY GOOD.

---

From: "Alan DeKok" <[email-masked]>
Date: Mon, 13 Sep 2004 13:04:57 -0400

Tony Finch <[email-masked]> wrote:
> extra hop in its journey:

  I understand, but that doesn't affect the SMTP server which performs
that validation.

  The multi-hop problem can be re-phrased as "any message must pass
all checks at all intermediate hops for it to reach the final
destination."  This restriction is nothing new to SMTP.  The specific
technique of authentication records in DNS is new.

> servers in order to whitelist them.

  And I don't see why (in the idealogical sense) this would be your
problem.  If other sites are forwarding your users messages, then
that's a matter between them and the users.  You shouldn't have to be
involved.

  Practically, the practice is common, and people resist mightily
changing their habits.

  We have to decide if the gain to some from performing MTA
authentication is worth the cost to others.  The people paying the
cost naturally don't want to do so.  That's what this WG is about.
Maybe there is an alternate implementation where you don't have to pay
the cost.  We should find it, if possible.

  If not, I don't see why your need to have forwarding for your users
should have any impact on my need to do what I want with my domain.

  Alan DeKok.

---

From: Tony Finch <[email-masked]>
Date: Mon, 13 Sep 2004 18:01:41 +0100

> that's a matter between them and the users.  You shouldn't have to be

If I naively start checking SPF records I will get involved because my
users will kick up a fuss about their legitimate email being rejected.

> should have any impact on my need to do what I want with my domain.

I'll feel free to ignore your bogus SPF policy then.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
WHITBY TO THE WASH: SOUTHWEST 5 TO 7. SHOWERS. GOOD. SLIGHT OR MODERATE.

---

From: "Alan DeKok" <[email-masked]>
Date: Mon, 13 Sep 2004 13:35:50 -0400

> users will kick up a fuss about their legitimate email being rejected.

  Your users may think it's legitimate, but if it doesn't pass SPF
checks, then the domain name they're using doesn't agree.

  Resolving that problem is an issue between those users, and the name
of the domain they're using.

> I'll feel free to ignore your bogus SPF policy then.

  Now you're getting personal.

  And I find it interesting that you admit you're not going to follow
the policies that other domains publish.  If this is true, I fail to
see why you're active in this group, as any standard reached here will
not be published, or checked, by you.

  It would also appear that you're opposed to the charter goals of
this group, or at the least, not interested in them.

  Alan DeKok.

---

From: Tony Finch <[email-masked]>
Date: Mon, 13 Sep 2004 19:58:29 +0100

> of the domain they're using.

That sounds like a statement that you are taking on the responsibility for
any problems that arise when you email a forwarding address.

>   Now you're getting personal.

No, I'll ignore everyone's :-)

> this group, or at the least, not interested in them.

I am interested in encouraging the group to agree on a standard that
doesn't break 5% of the legitimate email handled by my systems. From the
point of view of MTA authorization, I think CSV is the best solution
suggested so far. From the point of view of eliminating backscatter, I
think BATV is the best solution suggested so far. I am actively
contributing to both of these efforts. I am also working on extending BATV
into a mechanism that can eliminate envelope forgery far more effectively
and compatibly than SPF.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
NORTH FITZROY SOLE: WEST OR SOUTHWEST 6 TO GALE 8, VEERING NORTHWEST 5 TO 7.
SHOWERS. MAINLY GOOD.

---

From: Douglas Otis <[email-masked]>
Date: Mon, 13 Sep 2004 16:34:28 -0700

>   How does that affect MAIL FROM authentication via records in DNS?

There must not be assumptions made regarding mail channel integrity, so
there can not be a conclusion MAIL FROM has been authenticated.  The MTA
may be authorized to send on behalf of MAIL FROM, but that is not the
same as authenticating the entity associated with the identity MAIL FROM
as being the originator.

> DNS.  The previous hops are invisible to MAIL FROM checks.

Although such hops are apparent through the RFC2822 headers, both for
these headers and the RFC2821 MAIL FROM, there is no SPF or Sender-ID
mechanism to allow the authentication of their origin.  

> authentication records in DNS.

The current hop is authorized, not authenticated, to act on behalf of
the entity associated with the MAIL FROM identity.  SPF or Sender-ID
confirmation does not mean this entity associated with the MAIL FROM
identity had originated the message.  This would be making unverifiable
assumptions about the mail channel, whether the MTA is shared, and
whether the list was "open".  A check host function will not provide the
nature of the list.

>   For the recipient, nothing else is verifiable, and therefore nothing
> else is relevant.

The EHLO name is verifiable and high relevant when there is to be a
reputation assigned for abusive actions.  

> confusion somewhere in the conversation.

The assessment of reputation must be assigned accurately to the entity
acting as the cause of the abuse.  You are extending the concept of
authorization to be synonymous with having actually acted.   

> for the most inane reasons.

What does this mean?

> checking is perfectly valid, if all parties agree.

How do you arrive at publishing an SPF record being tantamount to
accepting accountability for all actions of these authorized entities? 
If one were to authorize a provider to send mail, and their provider's
system became compromised, would it not be better to accredit the
provider for resulting abuse?  Won't this resolve the problem sooner
than causing an innocent party to litigate for protection from a sloppy
reputation assessment?  In the case of an "open" record, this would hold
the SPF publisher accountable for any actions within the entire Internet
should they also expect to allow mail to be forwarded.

>   No... that's dumb.  "MTA's don't read mailing lists".

I assume there is an entity, meaning a sentient being able to read,
associated with the MAIL FROM identity.

> reasonable to expect that implementors of the standard have accepted

This extension of SPF authorization into also meaning acceptance of
accountability for the actions of authorized servers should be a warning
made in all capital letters!  I think such extension of accountability
rules out a safe use of any "open" list as well as any hope of allowing
forwarding and some list servers to function. 

> email scenarios may lead to such statements.

The marid-mpr draft shows how the authentication of the MTA client can
be accomplished separately from the mailbox-domain authorization.  By
doing both steps independently, there is no need to hold the mailbox
domain accountable for the actions of the client MTA they authorize. 
This allows the administrators of the mailbox domain to decide how they
wish to handle mail forwarding and list servers without the risk of
falsely obtaining a bad reputation.

> party?  Sure... that fits into the freedom, and the "end to end"

It does not require everyone to use a reputation service for such a
service to be effective in curtailing abuse.  Theses services assist
providers in locating problems and fending off enough load as to remain
viable.  There is nothing that mandates use of such services, but
adopting a universal means of accurately identifying the MTA client name
is vital for there to be a name based reputation service.  SPF and
Sender-ID is worthless with respect to identifying the MTA client. 
Making the authenticated EHLO visible within the MUA will provide
superior consumer protections over the PRA algorithm, and affords easier
compliance.

Start out with a nominal list useful for coloring mail.  If protocols
like a public version of BATV become prominent, then more may find it
reasonable to declare their lists comprehensive.

-Doug

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 13 Sep 2004 19:39:15 -0400

>   Practically, the practice is common, and people resist mightily

When the sender sends a message to the receiver, the edge of the
receiver´s system in this case would be the forwarder. It is the
responsibility of the receiver to make sure that the steps taken after
the forwarder accepts the mail from the sender´s system are correct.
Therefore in this specific case, the forwarder would have to do
Sender-ID checks in order to protect its users. Therefore, you don´t
have to do anything - it´s the end users that do. In a similar fashion
when a mailing list is involved, it is the responsibility of the list to
do the Sender-ID checks at the edge when it accepts the mail from its
users, and the end users will do their own checks against the list itself.

Yakov

---

From: "Alan DeKok" <[email-masked]>
Date: Tue, 14 Sep 2004 10:39:17 -0400

Douglas Otis <[email-masked]> wrote:
> The MTA may be authorized to send on behalf of MAIL FROM, but that
> is not the same as authenticating the entity associated with the
> identity MAIL FROM as being the originator.

  I agree 100%.

  The problem is that because the bounce path is not verifiable, the
originator is anonymous, and the MAIL FROM identity can't be trusted
for anything.  In the absence of a method to verify the bounce path,
authenticating the MTA using the domain in the MAIL FROM identity is a
good first step.

  I can think of scenarios where a domain sends mail from *only* one
MTA, and that MTA uses EHLO with another domain name.  In that case,
the first domain SHOULD have some way of saying "this MTA sends MAIL
FROM using my name".  Authenticating the EHLO doesn't get you that
information, so we need something more.

> SPF or Sender-ID confirmation does not mean this entity associated
> with the MAIL FROM identity had originated the message.

  But the recipient can't tell who the "real" originator is.  So far
as it seesm the SMTP client talking to it's SMTP server *is* the
originator.

> > for the most inane reasons.

  'publishers should also expect to be black-listed when they publish
an "open" record'

  Not because it's a good idea, but because people will do anything
they want, including blacklisting domains containing the letter "a".

  In other words, our desires about how the records are interpreted
may be written down in a specification, but we should not expect
everyone to follow that specification.

> accepting accountability for all actions of these authorized entities? 

  If that's what the spec says publishing a record means, that's the
semantics of the record.

> provider for resulting abuse?

  Multiple parties were involved, multiple parties share the
reputation gained through sending those messages.

> Won't this resolve the problem sooner than causing an innocent party
> to litigate for protection from a sloppy reputation assessment?

  That's a local legal issue.  Some jurisdictions have reasonable
laws, others don't.  We shouldn't let the possibility of legal action
dissuade us from designing a solution.

> domain accountable for the actions of the client MTA they authorize. 

  Interesting.  If I authorize someone to act on my behalf, I am
generally accountable for at least some of their actions.  Removing
this accountability is a unique feature of MPR.

  Alan DeKok.

---

From: Douglas Otis <[email-masked]>
Date: Tue, 14 Sep 2004 11:17:04 -0700

> authenticating the MTA using the domain in the MAIL FROM identity is a

You describe that as authenticating the MTA, but what this operation is
doing is validating the MTA as authorized to send a message containing
the MAIL FROM.  This identity has not been authenticated as originating
from the entity associated with the MAIL FROM.  

> information, so we need something more.

There are two things needed as independent steps.  One, authenticate the
name of the MTA to allow safe reputation assertions.  Two, determine the
MTA names authorized by the selected mailbox domain.  This can include
the MAIL FROM and the From mailbox domains.  These two steps then allow
for the reputation assertions and the blocking of spoofed mail for
either of these mailbox domains.  If the authorization list is only
nominal, then this list can be used to 'color' the mail at the MUA
rather than blocking it.

> as it seesm the SMTP client talking to it's SMTP server *is* the

But neither SPF nor Sender-ID properly identify the MTA.

> everyone to follow that specification.

This means one of two things with respect to SPF or Sender-ID. 

A) Remove all open list options.
B) Demand no reputations be based upon SPF/Sender-ID identities.

Option A breaks forwarding and several list servers.
Option B only helps reduce some of the spoofing, but does not allow a
named based reputation system using SPF or Sender-ID.

>   If that's what the spec says publishing a record means, that's the

That has not been made clear in the draft nor would allowing open lists
reflect this reality.

> reputation gained through sending those messages.

The multiple parties are the customers of the MTA provider.  By basing
reputation upon the provider by way of the EHLO name, a provider that is
not watching their logs or ensuring their server is secure, gets the bad
reputation.  This directly addresses the abuse problem by stopping the
source of the problem.  Blocking by the mailbox domain may unfairly
block innocent victims of spoofing, but blocking by the MTA EHLO name
will safely block the MTA with the problem.  If there is a mailbox
domain causing abuse, the provider must remove these problem accounts to
protect their reputation.  That is how it works today, but by using
names rather than IP addresses, history is easier to maintain.    

> dissuade us from designing a solution.

The solution should be able to accurately identify the source of abuse
as a means of avoiding these legal issues.  An email reputation service
must deal with the entire world. 

> this accountability is a unique feature of MPR.

This is establishing a hierarchy of accountability.  Those sending mail
must vet the accounts they relay to protect their reputation.  Those
holding accounts must then answer to their providers.  If a provider
fails to do their job, then as customers see their mail rejected, they
are free to find a different provider.  Providers will need their own
method to vet new accounts.  Neither Sender-ID nor SPF help in this
regard.

Sender-ID and SPF allows lax efforts of providers to hurt their
customer's reputation unfairly.  This model only seems to offer
litigation as a recourse for those unfairly harmed by this sloppy
reputation mechanism.

-Doug

---

From: "Alan DeKok" <[email-masked]>
Date: Tue, 14 Sep 2004 16:27:36 -0400

Douglas Otis <[email-masked]> wrote:
> the MAIL FROM.

  <sigh> The recipient (SMTP server) is authorizing the MTA (SMTP
client) as an MTA authentically within the domain named in MAIL FROM.

> name of the MTA to allow safe reputation assertions.

  And that's where I disagree.  For me, the point of MAIL FROM
authentication is not about reputation assertions, it's about the
ability of the domain named in MAIL FROM to control the use of it's
name.

  e.g. example.com & example.org share an outgoing MTA:
out.example.com.  In SMTP conversations, it announces itself as "EHLO
out.example.com".  It may then say "MAIL FROM: [email-masked]".

  Nothing in SMTP today permits the recipient to determine if
"out.example.com" is a valid outgoing MTA for example.org.  My goal in
this area is to provide a way to make that determination.  Stopping
spam, and reputation services, are interesting, but not something I
find to be practical.

  i.e. If MAIL FROM authentication records in DNS stop other people
using my name in vain, that's great.  I don't care how other people
restrict (or not) the user of their names: that's up to them.  For me,
the issue isn't that authentication will stop other people from
spamming me, it's that it will stop other people from claiming that
I'm spamming them.

> But neither SPF nor Sender-ID properly identify the MTA.

  For some meaning of the word "properly".

> Blocking by the mailbox domain may unfairly block innocent victims
> of spoofing, but blocking by the MTA EHLO name will safely block the
> MTA with the problem.

  You then label everyone at the MTA with a broad brush.  Sure, the
MTA is ultimately responsible, but if a recipient can use "MAIL FROM"
to tell that a subset of domains via that MTA are "bad", then it would
be imprudent to label *all* domains at that MTA "bad", just because
you authenticated the MTA via EHLO.

  Alan DeKok.

---

From: Douglas Otis <[email-masked]>
Date: Tue, 14 Sep 2004 15:23:11 -0700

>From the Latin authenticus- coming from the real author, of original or
first-hand authority.  It would be a mistake to apply this term to a
second-hand authority, as is the case with SPF or Sender-ID.  It would
be safer not to describe the real author (the mailbox domain at least)
as authenticated.  Rather, this mailbox domain has authorized
second-hand parties to act on their behalf and claim this authorization
has been validated.

> spam, and reputation services, are interesting, but not something I

If the EHLO name had been authenticated and the authorization validated,
as possible with CSV, then to assert which MTAs are authorized to send
mail on behalf of a mailbox domain only requires a very simple list of
PTR records.  This list can be obtained from a single DNS query and not
require any subsequent lookups to ascertain related addresses.  The
addresses will have already been checked by the "no-added overhead" CSV
operation. 

>From an overhead standpoint, this goal of stopping spoofed mail is
better done using two steps.  One is CSV, and the other is MPR.  CSV
does not add to the DNS overhead at all.  MPR only requires a highly 
deterministic single query to obtain authorization lists and the level
of the assertions.  This has the added advantage of not providing a spam
exploit where these authorization list are claimed to offer (bogus)
"authentication" of the MTA.  

>   For some meaning of the word "properly".

It is not proper to claim an MTA, authorized to send messages with
my-small-business mailbox domain, is the same as being administered by
my-small-business.com.  With SPF or Sender-ID, don't suggest
second-party servers for big-isp.com are administered by
my-small-business.com simply because an SPF or Sender-ID record granted
big-isp.com authorization to send messages.  To say that SPF or
Sender-ID authenticate big-isp.com as "being" my-small-business.com only
confuses the issue.  The risk from this muddled thinking is that these
identities may be employed to unfairly establish reputations.

> you authenticated the MTA via EHLO.

When the MTA administrator does not revoke access to abusive accounts,
why consider any message from this MTA to be valid?  The MTA may be
compromised or not checking SPF or Sender-ID records.  To block some
mailbox domain coming from a suspect MTA will also block this mailbox
domain from all other MTAs.  When the problem was with the MTA in
question, blocking the mailbox domain currently being spoofed will only
cause great harm to that domain and not stop the problem, as the abuser
will simply spoof the next mailbox domain (victim).

The reputation service is unable to determine what is being checked at
the MTA.  The reputation service does not know the source of the message
as you wish to imply.  The reputation service would be making a grievous
error assuming the MTA was being properly administered and the mailbox
domain was actually the source of the message.  Again, the mailbox
domain was not authenticated as you imply.  

-Doug

---

From: "Alan DeKok" <[email-masked]>
Date: Wed, 15 Sep 2004 10:13:35 -0400

Douglas Otis <[email-masked]> wrote:
> second-hand authority, as is the case with SPF or Sender-ID.

  I thought we were discussing security terms, not Latin.

> Rather, this mailbox domain has authorized second-hand parties to
> act on their behalf and claim this authorization has been validated.

  No.  The domain has published authentication information.  The
SMTP server then uses this information to decide if it wants to
authorize the SMTP client.

  The SMTP server cannot authenticate the client, because it doesn't
know who the client is.  The DNS server holding the records obtained
by the SMTP server cannot authorize the SMTP client, because the SMTP
client isn't asking that DNS server for permission to do anything.

  Security guards authorize people.  Corporate IT authenticates them.

> my-small-business.com.

  Which is why I never said that.

> > then it would
> why consider any message from this MTA to be valid?

  You're contradicting yourself.  Previously in this thread, you were
opposed to labelling people with a "broad brush".  Now, you're
claiming it's a good idea.

  Thanks.  This lets me know your position isn't well-defined.

  Alan DeKok.

---

From: Douglas Otis <[email-masked]>
Date: Wed, 15 Sep 2004 13:31:45 -0700

>   I thought we were discussing security terms, not Latin.

We are discussing the difference between send control lists and
authentication.

> authorize the SMTP client.

The domain has not published authentication information, as this
information is often not for hosts within their domain.  Should the
domain administrator wish mail forwarding to continue to operate, then
these send control lists must encompass the entire Internet.  How can
this list be described as authenticating anything?  In this case, it
would authenticate the entire Internet being within a domain!  This is
abusing the concept of authentication.

> know who the client is.

The SMTP server knows the SMTP client has claimed to be the EHLO name. 
I would think this would be a good starting point.

>   The DNS server holding the records obtained by the SMTP server
> cannot authorize the SMTP client, because the SMTP client isn't asking
> that DNS server for permission to do anything.

The SMTP client is asking to send a message.
  
>   Security guards authorize people.  Corporate IT authenticates them.

Send control lists authorize SMTP routes without authenticating the
identity permitted traversal.  Sender-ID or SPF offer an expensive forms
of routing constraints using address based send control lists.  When
1000 identities are authorized to traverse an SMTP route, a transaction
may safely carry identification of any of those authorized, depending
upon the unknown configuration of each MTA or the unknown nature of the
send control list.  As there are potentially many send control lists
which apply, even the total number of possible identities remains
unknown.  Constraining traversal in no way offers authentication of
identity.  None what so ever!
 
> opposed to labeling people with a "broad brush".  Now, you're

To refresh this point:
http://www.imc.org/ietf-mxcomp/mail-archive/msg04648.html

|>   Shared services means shared accountability.  If the recipient is
|> unable to distinguish between a "good" party on a shared MTA and a
|> "bad" party, then by definition, both parties fall into the same
|> classification.
|
| Sharing an MTA is a highly effective method used by ISPs to abate abuse.
| Reserve painting this technique with such a broad brush.  This wide
| brush never uncovers an MTA that may be compromised or poorly
| administered.  Holding those unable to take immediate corrective action
| accountable will cause a most unwelcome backlash.  The party that must
| be held accountable is the entity administrating policies of the MTA. 
| That party is only identified by an authenticated EHLO domain or the IP
| address.

You are advocating Sender-ID or SPF identities are authenticated.  I
still claim this is a misuse of the term authenticate as this presumes
the integrity of the mail channel and the nature of the list.  I noted
the hazard making this assumption would cause with respect to harming
reputations of those that, through no action on their part, receive
negative assertions.

There are valid reasons people share an MTA server.  I do not think it
is wise to suggest any message which traverses an MTA authorized for a
particular mailbox-domain should claim this MTA is (or is within)
"mailbox-domain."  Nor, when the message is spam, treat this event
"as-if" mailbox-domain had actually been authenticated.

This accountability must be seen as a hierarchy.  It is not fair or
effective to give mailbox domains a bad reputation when an MTA
administrator fails to check send control lists, or has an MTA that
becomes compromised.  Blocking the spoofed mailbox domain does not
correct the problem, but instead makes worse problems for those not in
control of the MTA causing the problem.  This would allow a spoofer to
single out a few domains at a time while not disrupting general use of
the problematic MTA.  The only recourse for those harmed in this fashion
would be through litigation.  The problematic MTA is never blocked.

>   Thanks.  This lets me know your position isn't well-defined.

At least it is not understood. : )

-Doug

---

From: "Alan DeKok" <[email-masked]>
Date: Wed, 15 Sep 2004 18:37:44 -0400

> We are discussing the difference between send control lists and

  Side-lines into :atin, and what words meant 2000 years ago aren't
helpful.  If they were, we wouldn't be using the letter "j", and we
would be pronouncing "v" as "w".

> http://www.imc.org/ietf-mxcomp/mail-archive/msg04648.html

  My point then, as now is:

  - Entities who have information should use it to make decisions.
  - Entities who do not have information cannot use it to make decisions.

> the integrity of the mail channel and the nature of the list.

  It presumes that the entities publishing those records have read the
specification of the protocol they're implementing, and understand
it's costs and benefits.  The meaning of those records is as defined
in the specifications, nothing more, or less.

> I noted the hazard making this assumption would cause with respect
> to harming reputations of those that, through no action on their
> part, receive negative assertions.

  Again, your focus is on reputations and negative assertions.  Mine
is not.

  http://www.imc.org/ietf-mxcomp/mail-archive/msg04818.html

  Your response to that message didn't acknowledge that my goals are
different from yours.  While I understand very well your concerns
about negative reputations, I do not see reputations as the primary
goal of SPF, Sender-ID, or the MARID WG.  Instead, what's important is
the ability of a domain to publish a list of authentic outgoing MTA's.

  Reputation systems are secondary to the primary goal, and are
piggy-backed on top of it.  I don't see a need to discuss MARID-based
reputation systems until such time as we have a clear specification of
not only an MTA authentication system using DNS, but also the costs
and benefits of using such a system.

  If that MTA authentication system is correct and useful, then
reputation systems can be built on top of it.  If the MTA
authentication system is not correct or useful, then any reputation
systems which depend on it will be likewise useless.

  Alan DeKok.

---

From: Douglas Otis <[email-masked]>
Date: Thu, 16 Sep 2004 12:02:43 -0700

On Wed, 2004-09-15 at 15:37, Alan DeKok wrote:
>   Side-lines into Latin, and what words meant 2000 years ago aren't

But through this period, the word has not changed meaning.

>   - Entities who have information should use it to make decisions.

The goal of this effort should also include providing essential
information to enable locating and correcting problems.  This would be
together with the other goal of detecting spoofed addresses.

>   - Entities who do not have information cannot use it to make decisions.

Ignorance is bliss? : )

> in the specifications, nothing more, or less.

I see a conflict even within the specification.  Jim Lyon has already
indicated any domain publishing an "open" record, requesting forwarded
mail receive a "Pass" assessment, will receive the bad reputation they
deserve.  The specification does not warn of this problem.  There is
another problem with any "open" record, even those that only request a
"Neutral" assessment.  SPF and Sender-ID use DNS records for the double
purpose of "authenticating" the MTA in conjunction with listing MTAs
authorized to send messages for a specific mailbox domain.  This
provides the MTA with the extremely weak identity of the mailbox domain.
When the record is "open" for SPF or Sender-ID, this invites spoofing of
the mailbox domain to obtain "authentication."  This invited spoofing is
very much counter to the goal of stopping the spoofing!

By first performing CSV to authenticate and authorize the MTA, and then
obtaining an authorization list of names referenced by the mailbox
domain, this will specifically prevent spoofing.  This two step approach
can not be exploited for the purpose of "authentication" to usurp the
reputation of the domain.  The weak "authentication" of Sender-ID or SPF
may be accomplished by simply asserting the mailbox domain in some
cases.

marid-mpr was to illustrate solutions for those interested in:
 a) Stopping spoofing
 b) Not breaking mail
 c) Providing fairly applied reputations
 d) Locating abusive MTAs
 e) Using lightweight and deterministic overhead
 f) Retaining network integrity
 g) Retaining DNS integrity
 h) Optional assertions on RFC2822 From that are not superseded
 
>   Again, your focus is on reputations and negative assertions.  Mine

My focus encompasses your concerns. If you review the marid-mpr draft
and consider how it can work, you will find it actually provides better
protections than SPF or Sender-ID, but without the overhead and the IPR
problems.

> the ability of a domain to publish a list of authentic outgoing MTA's.

The goal of authorizing the MTA is defined in the charter and made
somewhat explicit in the WG acronym.  By first identifying the MTA,
independent of any authorization, offers a few important benefits-

a) Stronger identity for asserting reputations    
b) Deterministic structure based upon names for authorization
c) Location of a problematic MTA
d) Prevention of record exploit and improved spoof prevention
e) Thwarting phishing when making the validated MTA identity visible
f) Retained mail functionality with safe "open" lists
g) A phase-in with open lists only "coloring" mail, not rejecting it

> and benefits of using such a system.

There should be some consideration given this, as you have indicated
your own expectation the results of this scheme will be used for
establishing reputations.  As Sender-ID and SPF provide such incredibly
weak identities, these identities will actually become more vulnerable
to harm caused by spoofing, but now from reputation services.  This will
happen due to an understanding SPF or Sender-ID strengthened these
identities.  As this is going from zero strength, the slight increase
and the surrounding hype make inappropriate reliance on these identities
far more likely.

> systems which depend on it will be likewise useless.

There are fundamental flaws with the SPF or Sender-ID scheme.  A list
may be "open" but such an open list is not made apparent to the
receiving MTA in all cases.  A sending MTA may be shared, but such
sharing is not apparent to the receiving MTA.  The Sending MTA may not
have made SPF or Sender-ID record checks upon receipt of the message
being sent, but this also is not apparent to the receiving MTA.  As a
result, there is validation promotion moving through these shared MTA
clients.  Essentially any domain accepted by the client MTA, may spoof
any other domain that also shares the client MTA and is using an "open"
list or where the sending MTA did not make the needed checks.

Even with full compliance to the specification, spoofing remains
possible.  This is creating a new danger where, had it been understood
not to trust the mailbox domain identity, there would be less risk.  The
hype used to promote these schemes suggests such identities can be
trusted.  Of course there are many ways to inject mail into the mail
channel that may bypass the needed checks, but where such an event can
not be detected using SPF or Sender-ID.  Both Sender-ID and SPF are
trusting the integrity of both the sending and receiving administrative
realms, but where there is no means to verify such assumption. 
Sender-ID and SPF are ideal mechanisms for the confidence artist.

There have been some that suggest any shared MTA should become a thing
of the past as a result of these flaws of Sender-ID or SPF.  These
shared MTA strategies, sometimes forced by blocking ports or by way of
transparent interception, are effective at curtailing abuse originating
within a providers networks and allows the provider to protect the use
of their IP address space.  This largely requires the examination of the
SMTP logs and finding too many nonexistent recipient errors.  This
shared MTA can also direct abuse@ messages to assist in the effort to
curtail abuse as well. 

I am not suggesting curtailing spoofing not be done.  I am simply
advocating a better, safer, cleaner approach.  And yes, it also makes
reputations safer as well.

-Doug

---

From: "Alan DeKok" <[email-masked]>
Date: Thu, 16 Sep 2004 16:32:42 -0400

Douglas Otis <[email-masked]> wrote:
> deserve.  The specification does not warn of this problem.

  The goal of specifications should not be to anticipate all possible
uses or abuses, of the specification.  To ask this of a specification
is to guarantee that it will fail to achieve consensus.

> domain, this will specifically prevent spoofing.

  I do not believe that there is enough consensus as to what "EHLO"
checking means, or "MAIL FROM" checking means.  Without that
consensus, we have no quantitative way of comparing proposals, or for
deciding if a proposal meets our requirements.

> establishing reputations.

  That is a matter for a follow-up WG, or a re-charter of this WG,
once this WG finishes its current work.  Until then, I see no point in
discussing the benefits or impacts of reputation systems.

> reputations safer as well.

  I'm not sure how any "MAIL FROM" checking can prevent spoofing on a
shared MTA.

  Alan DeKok.

---

From: "Scott Kitterman" <[email-masked]>
Date: Thu, 16 Sep 2004 17:23:42 -0400

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Alan DeKok
> Sent: Thursday, September 16, 2004 4:33 PM
> To: [email-masked]
> Subject: Re: SPF abused by spammers

>   I'm not sure how any "MAIL FROM" checking can prevent spoofing on a

I don't think it can.  That's why I give shared MTAs that I use a NEUTRAL
vice PASS result in my SPF records unless there are local restrictions to
prevent other customers of the MTA from using my "MAIL FROM".

What I can do with "MAIL FROM" checking is tell the world which MTAs are not
authorized through -all.

Personally, I would love to see some synergy between "MAIL FROM" and
SUBMITTER. If shared MTA operators would add SUBMITTER and an appropriate
2822 header field, then the result would be:

"MAIL FROM" - NEUTRAL (based on my domain)
SUBMITTER - PASS (based on the ISP domain).

This would mean, the domain owner says that the message is not certainly
forged and the MTA operator says it is accepting responsibility for the
message.  I would think that most people would accept this combination.  I
also think this is appropriate for shared MTAs because it is, in the end,
the operator of the MTA that is responsible for forgery it allows to happen.

Scott Kitterman

---

From: Mark <[email-masked]>
Date: Thu, 16 Sep 2004 21:50:14 GMT

Scott Kitterman wrote:

> What I can do with "MAIL FROM" checking is tell the world which MTAs
> are not authorized through -all.
>
> SUBMITTER. If shared MTA operators would add SUBMITTER and an
> appropriate 2822 header field, then the result would be:
>

I still stand by my earlier posted idea to use ENVID for this purpose, when 
I wrote:

> I was more than happy even using the existing ENVID for this purpose.
> ;) Something like:
>
> Mail from: <original@sender> ENVID=<forwarding@entity>
>
> Then we could have all used it tomorrow. I know, that is not what RFC
> 1891, section 5.4, meant it for; but it would have been fine by me. :)

Seriously, is there anyone out there who has ENVID currently in use? It just 
sits here, so unused. :)

Cheers,

- Mark

        System Administrator Asarian-host.org

---
"If you were supposed to understand it,
we wouldn't call it code." - FedEx

---

From: Douglas Otis <[email-masked]>
Date: Thu, 16 Sep 2004 15:52:15 -0700

> is to guarantee that it will fail to achieve consensus.

This still leaves anticipating the use of "open" lists provided by the
draft and its effect of "inviting" spoofing.  This aspect should be
within the scope of a draft that purports to solve a spoofing problem.

> deciding if a proposal meets our requirements.

There must be an exchange of ideas and concepts to arrive at a
consensus.  Announcing Caller-ID as the WG draft did not invite much
reflection other than largely the eventual removal of XML.  It would
seem a foregone conclusion more than consensus.  Publishing a draft
helps, but answering questions should provide clarity or offer a
perspective of the trade-offs considered.

As example, there were several serious issues never addressed.  The most
I heard with respect to network congestion avoidance, actually the lack
of avoidance, was "That would be bad."  The latest draft still retains a
required 10 script processing within 20 seconds minimum each.  The time
limit was added after it was pointed out valid record sets may take more
than a hour to resolve even with no packet loss.  The addition of the
time limit then fails to address a potential loss of network integrity
that may dramatically affect long haul links, or a potential for
flooding the network with UDP traffic.  A 3-5% packet-loss of UDP
packets is common but clearly not accommodated.  Ignoring this does not
make the problem go away.  Network considerations should be within scope
for a network protocol.  Plays well with others should be a requirement.

> discussing the benefits or impacts of reputation systems.

At the same time this group was declaring Last Call, there were
reputation services based upon the identities provided.  Statements of
being able to trust Sender-ID or SPF identities will not make these
identities trustworthy however.  There should be consideration given the
expectations created by hyperbola.   

>   I'm not sure how any "MAIL FROM" checking can prevent spoofing on a

The stronger the validation of a name associated with a message, the
less likely this validation will be spoofed.  As shown with Microsoft's
PRA algorithm, the name validated does not need to be the RFC2822 From
address.  Spoofing a Sender-ID or SPF check may only require asserting a
mailbox domain depending upon the nature of the list or the entry point
of the message.  CSV is not as easily spoofed, nor does it assault DNS
with a phalanx of variable records types to be resolved which may open
up other security risks as with SPF and Sender-ID.

Making this validated MTA name visible along with the RFC2822 From will
alert users to unexpected sources of the From mailbox domain based upon
the user becoming familiar with these identities during their normal
correspondents.  Unlike SPF or Sender-ID, the marid-mpr draft offers a
means for the domain administrator to specify these normal sources for
either the RFC2821 MAIL FROM or RFC2822 From mailbox domains that can
not be superseded as with Sender-ID.

The validation of the MTA name does not "invite" the spoofing of the
mailbox domain.  Use of a MAIL FROM constraint or the use of BATV will
also dissuade spoofing of this mailbox domain.  Institutions that find
themselves being phished or spoofed, should not employ outbound MTAs
outside their administrative realm and use a consistent identity for
their MTA names.  Unlike Sender-ID however, MPR can thwart spoofing of
the RFC2822 From as well as the RFC2821 MAIL FROM.  Such institutions
would need to alert their customers of this limitation, if their mail is
forwarded.  As MPR safely allows "open" lists, there can be a graceful
transition in making this restriction without "inviting" trouble.

-Doug

---

From: "Hector Santos" <[email-masked]>
Date: Thu, 16 Sep 2004 23:32:21 -0400

----- Original Message -----
From: "Alan DeKok" <[email-masked]>
To: <[email-masked]>
Sent: Thursday, September 16, 2004 4:32 PM
Subject: Re: SPF abused by spammers

Alan,

One of the key reasons I stayed away for the most part from this discussion
is because I don't think much progress will be made until RFC 2821
functional specification are modified to allow for enforcement and for
proper operations in a reduced and managed "spam world."

What is being proposed has many legal and operational ramifications and
until the RFC 2821 is modified, compliant legacy systems will continue to be
problematic, both technically and legally.

There are three (3) problems as I see it based on product operations:

1) The forwarding issue as we all know it,
2) US EPCA related issues regarding privacy, and
3) Malpractice and Bandwidth issues with Dynamic vs. Post SMTP analysis.

The industry has had two decades of standard email operational precedence
that 100% addressed #2 and #3. Systems were designed and engineered with the
corresponding ECPA laws in the books in mind.

It is my strong belief based on 20+ years of exclusive telecommunications
and electronic mail design and production, that Microsoft SenderID proposal
hits smack against the tradition with a major risk of creating many new
legal issues.

To summary the concerns:

The SUBMITTER concept will enforce SUBMITTER compliant ISP to operate in a
mode were users are no longer allowed to use alias email addresses.   This
has already been proved to be the case when the SPF compliant ISP has users
using alias email addresses and it failed at the SPF compliant MDA.
Microsoft's SUBMITTER will raise privacy concerns because the ISP must send
the user's main account with the mail transaction, an action the user many
not want.

The PRA concept will break optimized operations using dynamic SMTP
operations. It will enforce payload transmissions which has three critical
issues with it:

1) Risk operating in a mode where failure to delivery mail runs against the
US EPCA user rights for delivery expectations.

2) Introduces higher network bandwidth across all PRA compliant points.

3) Introduces higher and unnecessary bounce mail operations which is a major
factor in the distribution of malicious electronic mail.

It is for these key reasons, Santronics can not support the Microsoft
proposals based on its technical merits.  This does not take away the fact
that we made be forced to do so based on customer and market pressures,  but
it is not something we wish to spend resources on.  I firmly believe there
are better solutions and in my opinion, it begins with getting all the
issues with SMTP specifications squared away.

Once the industry, the world, has a new basis for SMTP operations, then all
the authentication and validations ideas make more sense and work more
reliably.  All future SMTP products will have "new rules" to work and more
importantly remains legal with the current laws in the books.

In my view, the following effort should be done:

1) SMTP specification refinement to clear all the ambiguities and specify
the new enforcement issues.

2) At a minimum,  introduce 2 new ESMTP commands:

TOPIC (or SUBJ to keep with the 4 letter standard)
HEAD

Both items solves the following:

1) It address optimized support for CANSPAM at the transport level
2) It address optimized support for Header Analysis at the transport level.
3) It helps the Microsoft PRA proposal.
4) It eliminates the payload problem.
5) It keeps the operation legal with no conflicts with the current EPCA
related laws in the books.

Note, none of this takes away from SPF or MARID,  but currently what is
being proposed does not help the process for all parties.  Since change is
required at some level for the SMTP server and receiver, we might as get it
right with the minimum requirements to address spam/spoofing.

Thanks

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office

---

From: Tony Finch <[email-masked]>
Date: Fri, 17 Sep 2004 15:53:04 +0100

>   I'm not sure how any "MAIL FROM" checking can prevent spoofing on a

For a rough outline, see http://www.cus.cam.ac.uk/~fanf2/hermes/doc/antiforgery/cam.txt

In summary: users submit email using SMTP AUTH. They are prevented from
using a bounce address that belongs to someone else (which is reasonably
easy to do with Exim). Their bounce address is modified according to BATV
or a similar scheme; the tag provides a guarantee that the message was
submitted by the authenticated user identified in the bounce address. Each
time a message is passed through the system for unauthenticated relaying
the bounce address is checked: if it's a local address it must be
correctly modified according to BATV.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
CAPE WRATH TO RATTRAY HEAD INCLUDING ORKNEY: SOUTHERLY 5 OR 6, DECREASING 4
FOR A TIME, BUT BACKING SOUTHEASTERLY AND INCREASING 6 TO GALE 8 OVERNIGHT.
RAIN LATER. GOOD BECOMING MODERATE. MODERATE OR ROUGH, OCCASIONALLY SLIGHT IN
THE MORAY FIRTH.

---

From: "Alan DeKok" <[email-masked]>
Date: Fri, 17 Sep 2004 13:06:01 -0400

> For a rough outline, see http://www.cus.cam.ac.uk/~fanf2/hermes/doc/antiforgery/cam.txt

  Quoting that sentence out of context, I can understand your
response.

  Your system describes a method for a shared MTA to guarantee that
any message it forwards is truly from the claimed originator.  That's
nice, but not useful to anyone else in the network.

  My repeated, and explicit, comments in this thread have described
the problems faced by the recipient who is sent the message from the
shared MTA.  That recipient has no way to verify that the message is
not spoofed by another domain using the shared MTA.

  Any method which permits the recipient to verify the "MAIL FROM"
must involve the originator.  The shared MTA can claim that it is
forwarding the message, but because it is not the originator, it
cannot authenticate the message as truly coming from the originator.

  The shared MTA can use special/local information in order to
authenticate to it's own satisfaction the "true" origin of the
message.  That information is not available to others in the network,
so they cannot perform the same authentication to satisfy themselves
as to the true origin of the message.  And they cannot take the word
of the shared MTA, as it may be compromised.

  Alan DeKok.

---

From: Tony Finch <[email-masked]>
Date: Fri, 17 Sep 2004 18:39:40 +0100

> nice, but not useful to anyone else in the network.

My setup's a bit odd in that the MX, MSA, and general-purpose relay are
different functions of the same system. This means care is required when
thinking about the various functions.

Although you can set up BATV on an outgoing relay in the manner that you
rightly criticise, if you set it up on an authenticated MSA and the
corresponding MDA you do get the guarantees that you want.

> must involve the originator.

Agreed.

> The shared MTA can claim that it is forwarding the message, but because
> it is not the originator, it cannot authenticate the message as truly
> coming from the originator.

In my case I can because the shared MTA is also the MSA and the final
delivery system for the domain in question. It is therefore authoritative
regarding which addresses are valid at that domain. If a message is
submitted other than via the MSA, or without a username and password
corresponding to the bounce address, it'll be rejected.

> That information is not available to others in the network, so they
> cannot perform the same authentication to satisfy themselves as to the
> true origin of the message.

At the moment you can use callback verification to acheive this. We hope
to have more lightweight alternatives in the future. Note that if
intermediate MTA relays do CBV then the MSA/MDA for the domain don't have
to be directly accessible to the Internet: CBV is transitive.

> And they cannot take the word of the shared MTA, as it may be
> compromised.

At that point you have to rely on reputation systems. Authentication is
not the whole of the solution. A domain with poor authentication is a
spammer-friendly domain.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
COLWYN BAY TO THE MULL OF GALLOWAY INCLUDING THE ISLE OF MAN: SOUTH OR
SOUTHWEST 4 OR 5, INCREASING 6 TO GALE 8 FOR A TIME, VEERING WEST 5 OR 6
LATER. RAIN THEN SHOWERS. MODERATE OR GOOD, PERHAPS POOR FOR A TIME. ROUGH.

---

From: "Chris Haynes" <[email-masked]>
Date: Fri, 17 Sep 2004 20:18:45 +0100

"Alan DeKok" declared:
<snip>
> of the shared MTA, as it may be compromised.

My understanding is that SPF records declare the policy of the _sender_.  If the
sender trusts the shared MTA to verify all originators and to prevent
cross-customer spoofing, then the sender can use something like '+mx -all' and
the receiver should respect the sender's trust in the shared MTA s/he uses.

If the sender does not trust the shared MTA, the s/he will be ill-advised to
publish such a self-confident policy.

If the sender's trust in the shared MTA is misplaced, IMHO that's an issue for
the sender, not for the receiver.

I think the more pressing  issue in this area is a deployment one:

How many MTA 'products' are there today which would let a well-intentioned MTA
service provider implement and operate such a shared MTA?

How many service operators have now, or have plans for, such a service?

When this question was asked recently asked in 'another place' - one very
interested in SPF - there were two declarations of intent to offer what I would
term an 'alpha-grade' service in a few weeks time. No one could identify any
production-grade services or technology.

I'd be delighted to see evidence to the contrary.

This lack of a suitable shared-server technology could limit the rate of market
deployment of whatever scheme we chose - particularly for those many domains
unable / unwilling to run their own dedicated outbound servers.

Chris Haynes

---

From: "Alan DeKok" <[email-masked]>
Date: Fri, 17 Sep 2004 16:54:30 -0400

"Chris Haynes" <[email-masked]> wrote:
> My understanding is that SPF records declare the policy of the
> _sender_.  If the sender trusts the shared MTA to verify all
> originators and to prevent cross-customer spoofing, then the sender
> can use something like '+mx -all' and the receiver should respect
> the sender's trust in the shared MTA s/he uses.

  In that case, the recipient has checked with the originator, and has
information by which to decide whether or not to trust the shared MTA.

  And yes, "originator trusts the shared MTA" still may mean that "the
message may be spoofed".  This indicates a weakness in any MAIL FROM
authentication. when shared MTA's are used.

  Alan DeKok.

---

From: "Chris Haynes" <[email-masked]>
Date: Fri, 17 Sep 2004 22:37:46 +0100

> information by which to decide whether or not to trust the shared MTA.

Yes - that's the way SPF works as it uses Mail-From.

> authentication. when shared MTA's are used.

When I say "originator trusts the shared MTA" I mean "trusts the shared MTA to
have authenticated and identified the sender of each message (using SMTP AUTH or
equivalent),  and to have applied whatever logic is needed to prevent spoofing,
i.e. to ensure, for every message, that the sender is authorised to use the
'Mail-From' of that message".

If the originator's trust in the MTA' s ability to prevent spoofing is
well-founded, than the message cannot be spoofed, and Mail-From can be trusted.
(barring IP address spoofing, DNS penetration, physical penetration of MTA etc.
etc. - which are factors to be listed in the "Security Considerations").

I can't understand the basis of your concern about a "weakness in any MAIL
FROM..." .

Chris

---

From: Douglas Otis <[email-masked]>
Date: Fri, 17 Sep 2004 16:33:38 -0700

On Fri, 2004-09-17 at 14:37, Chris Haynes wrote:
> i.e. to ensure, for every message, that the sender is authorized to use the

When the SPF list is not terminated with "-all" but rather "?all" to
ensure forwarded mail receives a "Neutral" assessment and is not lost,
then even a shared MTA checking incoming mail for compliance with SPF
records allows a different domain with access to have their spoofing
message promoted to a "Pass" assessment.  In this case, the MTA provider
is in full compliance with the SPF specification, but "Pass" level
spoofing only requires a simple assertion of the mailbox domain.

An "open" list allows mail to continue normal functionality.  Until
public versions of digitally signed mail, such as IIM, DomainKeys, or
BATV are available, there is nothing to fill the gap to assure the
sender their mail will not be rejected based upon policies that describe
their nominal outbound mail but then make no other allowance.  Many
providers will not want their customer's mail rejected and cause a
disruption in service.  Their customer support efforts will be large and
likely met with an attrition.  Forcing use of a mailbox domain is also
counter to consumer protection as this needlessly ties them to a
provider.

Beyond digital signatures, there is nothing to avoid this "closed" list
problem.  The marid-mpr mechanism, instead of Sender-ID, stops inviting
>From spoofing as a means of obtaining a false reputation.  The use of
BATV also stops inviting the spoofing of MAIL FROM as a means to avoid
access blocking.

The weakness in using either MAIL FROM or the PRA for establishing an
identity used in a reputation system is caused through the ease of
spoofing this identity.  In addition, this identity fails to locate the
problem as a means to curtail abuse from either the sender's or
receiver's perspective.  By using the stronger MTA name for performing
reputation services, a compromised or poorly administered MTA is located
and blocked.  This imposes a responsibility on the providers to monitor
their outbound SMTP logs and respond to abuse@ complaints.

-Doug

---

From: "Alan DeKok" <[email-masked]>
Date: Sat, 18 Sep 2004 11:41:24 -0400

"Chris Haynes" <[email-masked]> wrote:
> When I say "originator trusts the shared MTA" I mean "trusts the
> shared MTA to have authenticated and identified the sender of each
> message (using SMTP AUTH or equivalent),

  The recipient can verify that the originator intends the shared MTA
to use "MAIL FROM" it's domain.  The recipient cannot tell if any
individual message sent by that shared MTA is actually from the
alleged originating domain.

  The "trust" you mention above includes two separate trust
relationships, only one of which can be verified by the recipient.  To
me, this means we should talk about the trust relationships in
isolation, and not together.

  It also means that a binary trust value for the shared MTA is
inadequate.  A "trusted" value can be used for MTA's you control, an
"untrusted" value for MTA's you don't like, and a "maybe" value could
be used for shared MTA's.  As noted earlier, this is being done today
with SPF.

> I can't understand the basis of your concern about a "weakness in any MAIL

  I'm looking at it from the view of the recipient, and from an
examination of what information is available, and what things are
being trusted.  We can say that publishing a "MAIL FROM"
authentication record has the meaning you intend, but that meaning
contains pieces ("trusted" message origin) that contains claims by the
originator which cannot be verified by the recipient.  In contrast,
the "trusted to use MAIL FROM with my name" claim CAN be verified by
the recipient: he has a connection where the shared MTA is using MAIL
FROM with that name.

  My method is a bit reductionist: divide everything into small
pieces, and examine how those pieces are used, and how they're put
togther.  Only when we understand the pieces can we be confident that
a system built on those pieces does what we want.

  The alternative is to build a "house of sand".  It's pretty, but
with no foundations.  It will collapse as soon as one of the unspoken
assumptions is questioned, or proven wrong.

  Alan DeKok.

---

From: "Chris Haynes" <[email-masked]>
Date: Sat, 18 Sep 2004 18:05:30 +0100

> alleged originating domain.

The recipient is invited to accept that the message _is_ from the purported
domain because the Sender's Policy says he should - she has declared that shared
MTA trustworthy. It's her message, her policy, her choice of shared MTA.

When the recipient is offered the message he extracts the domain of the
Mail-From and consults the DNS for that domain to get the policy. He then
applies the policy 'algorithm' to the IP address of the sending MTA to see if it
was authorised by the sender, and, if so, what the 'strength' of the sender's
policy is on this particular MTA.

If her policy says 'pass' ('+'). then she is saying 'If this MTA offers you a
message purporting to be from my domain, I trust it, and so should you'.

The receiver has to trust:

1) The DNS service to deliver the correct SPF policy for the domain

2) The IP transport system to report accurately the IP address of the sending
MTA.

There his need to trust anything associated with the shared MTA ends.

The message has been identified as coming from the sender's domain, and the
sender has accepted responsibility for the behaviour of the MTA.

> isolation, and not together.

I invite you to be specific about this second trust relationship you talk of.
Just what is it that the reciever has to trust the sending MTA to do or not do,
and which is not covered by the sender's trust in that MTA?

A specific example would help me understand your concern.

> be used for shared MTA's.  As noted earlier, this is being done today

Agreed - I don't think I was advocating a 'binary' value in the trust itself,
was I?

> a system built on those pieces does what we want.

That's what I have started above.

> assumptions is questioned, or proven wrong.

If you intend this as a criticism of SPF then you have to expose these unspoken
assumptions.
Please be specific.

Let me explain where I'm coming from:

I'm not associated with any one scheme.  I'm a 'sufferer' who needs a workable,
engineeringly-viable solution.

I see in SPF something that's pretty neat - I like its architecture, the one
which keeps the sender in charge of the policy. But I can also see problems with
it, e.g. with whether the _sender's_ trust in an MTA not under her
administrative control can ever be well-founded.

There appear to be several other schemes being proposed in this space.  At the
time of the chairs' choosing I'll be just as interested in them.

I think those who have alternative schemes to offer are being put in a difficult
position by the procedure that has been adopted in this WG.  We are being
invited to consider the candidates one by one.  This goes agains all my
engineering instincts, which are to:

a) Agree the objectives and metrics of success,

b) Identify all viable candidates,

c) Assess them in parallel against the objectives, constructing interesting
scenarios, test cases etc. - against which all candidates can be evaluated.

Instead we have a procedure which feels as if it could end with a winner being
declared as soon as the first 'just about acceptable' solution has been
identified.

Those proponents of candidate solutions which have not yet been considered
probably feel the need to continually draw attention to the defects of those
candidates which were lucky enough to be considered first.  This is challenging
because the points of difference are difficult to describe when you are not
allowed to be specific about  the 'better' approach which you cannot yet talk
about.

So I have a lot of sympathy for those proponents waiting to get their candidate
exposed.

But I, for one, feel that any criticism of a scheme which _is_ on the table has
to be phrased in very concrete terms. There has to be a specific example of a
weakness, of a security vulnerability etc. for the rest of us to be able to
understand the concern.

It may well be that you can see some specific weakness in the use of Mail-From
that the rest of us cannot see and that you can do the WG a service by exposing
that specific weakness.  I've been trying to manoeuvre you into doing this with
your 'trust' concern.  I'm genuinely trying to understand the 'space' in which
the receiver's trust or mistrust of the sending MTA matters to you.

What is it that the sending MTA can do or not do which, whilst still honoring
the trust that the sender has in that MTA, causes Mail-From to be flawed?

Chris Haynes

---

From: "Alan DeKok" <[email-masked]>
Date: Sat, 18 Sep 2004 22:04:33 -0400

"Chris Haynes" <[email-masked]> wrote:
> The recipient is invited to accept that the message _is_ from the
> purported domain because the Sender's Policy says he should - she
> has declared that shared MTA trustworthy. It's her message, her
> policy, her choice of shared MTA.

  I can publish a record saying my MTA is the best in the world.
That's my perogative as admin, but it doesn't mean that statement is
an accurate refelection of reality.  It's a statement of faith, and
nothing more.

> 2) The IP transport system to report accurately the IP address of the sending

  The recipient already trusts DNS & IP transport, independent of any
additional records in DNS.  Adding SPF or other records in DNS changes
nothing about that trust.

> sender has accepted responsibility for the behaviour of the MTA.

  Which is not the same as claiming the message actually originating
from the sender.  So why make that claim?

  Instead, claim that "MAIL FROM" checks means that the domain admits
that the shared MTA can use it's name in MAIL FROM, and that the
domain will accept the bounces from messages sent by that MTA, for
messages which use it's name in MAIL FROM.

> I invite you to be specific about this second trust relationship you talk of.

  I'm not sure what part of my message was unclear.

  a) An "originating" domain allows the shared MTA to send messages
     using it's name in "MAIL FROM".  The recipient can verify that
     the name is being used, and that the "originating" domain has
     publicly stated that such use of it's name is permitted.

  b) An "originating" domain claims that when a shared MTA uses it's
     name in "MAIL FROM", that those messages are not being spoofed by
     another party sharing that MTA.  The recipient has no way of proving
     to his own satisfaction that such a statement is true.

     Therefore, that statement has little or no meaning the recipient.
     It is useful solely as a statement about the "originating"
     domain's beliefs, and not as a statement reflecting any kind of
     objective reality.

> Agreed - I don't think I was advocating a 'binary' value in the trust itself,

  No.

> If you intend this as a criticism of SPF then you have to expose
> these unspoken assumptions.

  It's not a criticism of SPF.

> I'm not associated with any one scheme.  I'm a 'sufferer' who needs
> a workable, engineeringly-viable solution.

  That's pretty much my stance.  I haven't seen anything dramatically
useful, just a collection of tools which can help.

> We are being

  The charter already talks about MTA authorization records in DNS.
To a certain extent, any discussion of the objectives is too late.

> scenarios, test cases etc. - against which all candidates can be evaluated.

  Yes, well, that would be nice.  Instead, we have umpteen patent
discussions, and few ways to determine which proposal is "better", or
even if a proposal is "good enough".

> It may well be that you can see some specific weakness in the use of
> Mail-From that the rest of us cannot see and that you can do the WG
> a service by exposing that specific weakness.

  I'm not sure it's a weakness, so much as a lack of information.  The
MAIL FROM isn't about the originator, it's about the bounce path.  So
all you know about any value used in MAIL FROM is:

  a) the SMTP client sent you that value
  b) that value might be an actual bounce path for the message

  Any "trust" in the MAIL FROM cannot be based on information which
MAIL FROM does not contain: the identity of the originator.  Any
"trust" in MAIL FROM can only be based on the information which MAIL
FROM does contain: the bounce path.

>  I've been trying to manoeuvre you into doing this with your 'trust'
> concern.  I'm genuinely trying to understand the 'space' in which

  I understand, and I've tried to explain as best I can.

> What is it that the sending MTA can do or not do which, whilst still
> honoring the trust that the sender has in that MTA, causes Mail-From
> to be flawed?

  It's not the MAIL FROM which is flawed, it's the ability of the
recipient to believe the senders trust in the shared MTA, as anything
other than a statement of faith made by the vender.

  Faith is nice for the believer, but it means nothing to someone
else.  Experiments, and directions for reproducing those experiments,
are useful to both the experimentor and to anyone he's trying to
convince.  For MARID, unverifiable statements about a senders beliefs
are faith, nothing more.

  Or, as Emily Dickinson said:

	Faith is a fine invention
	When gentlemen can see,
	But microscopes are prudent
	In an emergency.

  Alan DeKok.

---

From: "Chris Haynes" <[email-masked]>
Date: Sun, 19 Sep 2004 10:57:22 +0100

> other than a statement of faith made by the vender.

Ah! Now here we can agree. We may differ as to _whose_ trust is broken, but the
crucial point is that a shared MTA is being trusted to have ensured that the
sender is authorised to use the Mail-From. There is no indication in the SMTP
protocol, or in the message headers, that this authorisation is actually taking
place.

I raised this problem as a criticism of PRA - where the MUA had no indication
that the MTA had actually verified the PRA.

Exactly the same criticism can now be directed at the current set of drafts.

All sorts of operational errors, changes in policy, etc., could mean that the
tests at the shared MTA have temporarilly ceased, yet the node is still being
trusted.

This should be of particular concern to those senders who use a shared outbound
MTA not under their administrative control (e.g. customers of most ISPs).

I've just posted a new thread "Proposal for VERIFIED keyword".

Would something like this help your trust concerns?  It would mine.

Chris Haynes

---

From: "Mark C. Langston" <[email-masked]>
Date: Sun, 19 Sep 2004 10:57:50 -0700

> protocol, or in the message headers, that this authorisation is actually taking

But with SPF, the trust you're being asked to place is not whether the
sender is authorized to use that MAIL FROM:; it's whether the entity
connecting to your MTA (the destination MTA, presumably) is one
associated with the MAIL FROM: RHS.  It's a somewhat subtle, but
important, distinction.

-- 
Mark C. Langston            GOSSiP Project          Sr. Unix SysAdmin
[email-masked]   http://sufficiently-advanced.net    [email-masked]
Systems & Network Admin      Distributed               SETI Institute
http://bitshift.org       E-mail Reputation       http://www.seti.org

---

From: "Chris Haynes" <[email-masked]>
Date: Sun, 19 Sep 2004 20:05:42 +0100

> associated with the MAIL FROM: RHS.  It's a somewhat subtle, but

I beg to differ.

SPF can check the whole of the MAIL-FROM address, not just the RHS.  You can
construct policies which use the macro function to insert the LHS into a further
look-up - thus giving a policy which is user-specific, not just domain-specific.

A 'trusted' shared MTA must prevent cross-customer forgery to at least the level
of granualarity of the corresponding SPF policy.  So, to be _sure_ of providing
the level of trust required in offering the message it must, I think, ensure
that the whole Mail-From address LHS@RHS may be used by whatever entity is
asking it to send the message.

My language "sender is authorised to use..." relates to the tests which a
trusted MTA server needs to undertake internally to discharge the trust that is
placed in it, not to the semantics of the SPF test itself.

Chris Haynes

---

From: Dave Crocker <[email-masked]>
Date: Sun, 19 Sep 2004 12:29:42 -0700

Mark,

>  associated with the MAIL FROM: RHS.  It's a somewhat subtle, but

I think that summarizing SPF as asking "whether the sender is authorized to use 
that MAIL FROM:"

Rather than simply giving a different summary that is valid, could you please 
explain what is wrong with the semantics of this one?

d/
--
Brandenburg InternetWorking
[email-masked]
+1.408.246.8253

---

From: "Mark C. Langston" <[email-masked]>
Date: Sun, 19 Sep 2004 12:39:35 -0700

> explain what is wrong with the semantics of this one?

Yes.  Sorry for not being clear the first time.  It's a problem for the
reason pointed out by the poster to whom I was responding:  it says
nothing about the policies of the entity connecting to the MTA doing the
SPF check.  There's no way to know whether the original sender was, in
fact, a legitimate claimant to that MAIL FROM:.  The connecting MTA
could be an open relay, allowing anyone to claim that MAIL FROM: and
pass it through that MTA to the destination.  If the SPF record checks
out, the best you can say is that the connecting MTA is legitimately
associated with that domain name.

---

From: Dave Crocker <[email-masked]>
Date: Sun, 19 Sep 2004 13:30:52 -0700

On Sun, 19 Sep 2004 12:39:35 -0700, Mark C. Langston wrote:
>      it says
>  SPF check.  

Let's see:

	-  The MailFrom contains a domain name.  
    -  Under that domain name, the client MTA is registered.
    -  The client MTA does not have to give the domain name administrator 
       "permission"
    -  The client MTA *does* decide to carry the message containing that 
       MailFrom

So there really *is* a policy relationship between the creator of the MailFrom 
and the operator of the MTA.  (An open relay is merely an MTA with a promiscuous 
policy.)

> There's no way to know whether the original sender was, in

Oh.  You are saying that there is a formal relationship between the MailFrom 
domain and the client MTA, but no enforcement of any relationship between the 
Mailfrom and the RFC2822.From or RFC2821.Sender?  And, in fact, From or Sender 
can specify any MailFrom they want?  Good point.

However, the MailFrom domain presumably register client MTA addresses that it 
trusts.  If the MailFrom domain administrator is registered addresses for open 
relays, that's their problem.

In other words, the presumption is that the MailFrom domain very much is making 
an explicit decision about the acceptability of that client MTA.

So, when the original sender chooses to go through a particular MTA, using a 
particular MailFrom, the presumption is that the administrator of that MailFrom 
trusts the posting policies of the client MTA.  Presumably this means that the 
client MTA has appropriate controls on who can post.  This would mean that the 
kind of possible MailFrom abuse you are suggesting would not be permitted by 
that Client MTA.

So, this still sounds to me like " the sender is authorized to use that MAIL 
FROM:", albeit through a pretty obscure set of relationships.

d/

---

From: Douglas Otis <[email-masked]>
Date: Sun, 19 Sep 2004 15:14:30 -0700

On Sun, 2004-09-19 at 12:05, Chris Haynes wrote:
> > > >   It's not the MAIL FROM which is flawed, it's the ability of
> > > > the recipient to believe the senders trust in the shared MTA, as
> > > > anything other than a statement of faith made by the vendor.
> > >
> > > Ah! Now here we can agree. We may differ as to _whose_ trust is
> > > broken, but the crucial point is that a shared MTA is being
> > > trusted to have ensured that the sender is authorized to use the
> > > Mail-From. There is no indication in the SMTP protocol, or in the
> > > message headers, that this authorization is actually taking place.
> >
> SPF can check the whole of the MAIL-FROM address, not just the RHS. 
> You can construct policies which use the macro function to insert the
> LHS into a further look-up - thus giving a policy which is
> user-specific, not just domain-specific.
> 
> A 'trusted' shared MTA must prevent cross-customer forgery to at least
> the level of granularity of the corresponding SPF policy.  So, to be
> _sure_ of providing the level of trust required in offering the
> message it must, I think, ensure that the whole Mail-From address
> LHS@RHS may be used by whatever entity is asking it to send the
> message.
> 
> My language "sender is authorized to use..." relates to the tests
> which a trusted MTA server needs to undertake internally to discharge
> the trust that is placed in it, not to the semantics of the SPF test
> itself.

There are a few security concerns using a script that can reference
labels for a third parties (victims) to resolve in sequence.  The
ability to direct lookups to one DNS server and then another will
identify the source port for some implementations.  This then makes a
birthday attack relatively simple.  Using this label construction
macro's ability to resolve valid users is also fraught with additional
security problems.  In addition with a need to create a replicate user
list for each IP address used by the outbound MTA, the NXT function will
allow this list to be discovered.

As was illustrated by the marid-mpr draft, multiple MTA domains can be
associated with a mailbox domain without requiring these dangerous
subsequent DNS lookup on different servers.  The TXT script approach may
already overwhelm the DNS resolver.  To include a function normally
performed by the source domain only seems to be in denial of the
problem.  The dangerous macro language allows TXT, A, AAAA, MX, and PTR
records to be queried using arbitrary labels.  This seems like a really
bad idea and one nearly impossible to defend, especially when the use is
to grant authorizations.

-Doug