Subject: draft-schlitt-spf-classic-02.txt

From: Douglas Otis <[email-masked]>
Date: Wed, 08 Jun 2005 12:15:12 -0700

1.  Introduction
...
,----
| An additional benefit to mail receivers is that after the use of an
| identity is verified, local policy decisions about the mail can be
| made based on the sender's domain, rather than the host's IP address.
| This is advantageous because reputation of domain names is likely to
| be more accurate than reputation of host IP addresses.  Furthermore,
| if a claimed identity fails verification, local policy can take
| stronger action against such e-mail, such as rejecting it.
`----

2.5.3.  Pass
,----
| A "Pass" result means that the client is authorized to inject mail
| with the given identity.  The domain can now, in the sense of
| reputation, be considered responsible for sending the message.
| Further policy checks can now proceed with confidence in the
| legitimate use of the identity.
`----

This is where SPF and Sender-ID make a very serious mistake.  This
assumes "server authorization" is equivalent to "sender authentication."
It would be like me making a declaration that the postal service is
authorized to deliver my letters, where recipients are then claiming any
letter received from the postal service bearing my name is authentically
or genuinely from me.  Of course that would be a false assumption, yet
this draft describes the sender as "considered responsible for sending
the message."  Review the many assumptions being made before arriving at
this conclusion.  This false assumption is also why publishing SPF
records is unwise for the majority of domain owners.

2.1.  The HELO Identity
,----
| The "HELO" identity derives from either the SMTP HELO or EHLO command
| (see [RFC2821]).  These commands supply the SMTP client (sending
| host) for the SMTP session.  Note that requirements for the domain
| presented in the EHLO or HELO command are not always clear to the
| sending party, and SPF clients must be prepared for the "HELO"
| identity to be malformed or an IP address literal.  At the time of
| this writing, many legitimate e-mails are delivered with invalid HELO
| domains.
|
| It is RECOMMENDED that SPF clients check not only the "MAIL FROM"
| identity, but also separately check the "HELO" identity by applying
| the check_host() function (Section 4) to the "HELO" identity as the
| <sender>.
`----

There is an expression, when your only tool is a hammer, everything
looks like a nail.  While checking the HELO may extend the use of name
reputation into the initial exchange, the use of SPF to confirm the HELO
is using a rather heavy hammer.  The need for checking the HELO would be
to protect network resources using a unified name reputation service.
The potential for hundreds of DNS queries to resolve something as simple
as HELO means this scheme has a rather basic flaw.

While there is no reason for SPF to define how results are tallied,
effectively this check could double an already sizeable overhead.  Would
the HELO override the MAILFROM?  With respect to finding a means to stop
abuse at the source, the HELO would prove more useful.  However, SPF is
poorly suited to fulfill this function.

The prior justification for attempting to resolve the HELO was limited
to the handling of a DSN message with a null MAILFROM.  With SPF failure
expected in this case, wouldn't something like BATV be more effective in
this situation?

2.4.  Checking Authorization
...
,----
| Without explicit approval of the domain owner, checking other
| identities against SPF version 1 records is NOT RECOMMENDED because
| there are cases that are known to give incorrect results.
`----

Do you really think this sentence will cause a company to change their
released version of Sender-ID?  Sender-ID advocates will be quick to
indicate they use both methods looking for any authorization.  This
becomes potentially a disaster when reputation is then misapplied, due
to there being a lack of sender assurances.  If you insist at retaining
the v=spf1 record, providers should expect needing a license to assure
the PRA.

I fail to see the justification for not offering a positive means to
express the scope of the record.  Depreciate the version 1 record.  Both
SPF "new classic" and Sender-ID may continue to utilize these older
records.  Moving forward, a scoped version of the record offers a safer
approach.  It would also permit SPF records to be used for white-listing
only.

There have been many technologies thwarted by a large company's ability
to dictate the definitions they promulgate.  A large company may not be
right or fair about their selection.  Something as simple as data
compression caused sizeable losses by a last minute imposition, even
when eventually adjudicated as unfair.  Accept this banal reality.

-Doug

---

From: Aredridel <[email-masked]>
Date: Wed, 08 Jun 2005 18:20:40 -0600

> records is unwise for the majority of domain owners.

If there were only one post office, not many millions, the analogy might
be more accurate.

It's not entirely firm authentication, but it's sure better than having
no information at all. . . But would you take less than fully secured
crypto, or is the perfect the enemy of the good in this case?

Ari

---

From: Douglas Otis <[email-masked]>
Date: Wed, 08 Jun 2005 19:02:25 -0700

> If there were only one post office, not many millions, the analogy might

There is certainly more domains than mail servers.  If the average
server were to handle just two domains, this still leaves considerable
room for disputes to arise.  From the perspective of a reputation
service, it is impossible to know either the number of domains sharing
the server, or what identifiers were assured.  Reputation can be very
destructive when attributing abuse to the wrong entity.

> crypto, or is the perfect the enemy of the good in this case?

SPF does not offer authentication.  Publishing this SPF authorization
record can be much worse than not publishing, when used beyond just
white-listing.  With spammers and phishers already among the first to
offer SPF records, SPF authorization alone provides no protection.  When
reputation is applied to authorization, disaster!

SPF/Sender-ID is server authorization.  Because it is untrustworthy, (it
causes email to be lost or refused) SPF/Sender-ID has extremely little
value.  It can not be applied in any strict manner reliably.  The high
overhead associated with resolving the path registered by SPF, means
this mechanism can not offer network protection.  Nor can SPF reliably
offer recipient protection either.  Phishing and spamming is sure to
continue with SPF/Sender-ID.

I know there are some that dream of the day when "-all" does not cause
mail to be lost, and when everyone knows which identifier to check. SPF
demands additional diligence by providers, while also leaving the
providers unscathed when they are negligent. SPF may cause a behavior
change in some cases, but this change is almost certain to create even
greater damage for domain owners.

Something like DomainKeys, that will eventually be upgraded to DKIM,
does offer an alternative that is both safe and fair _authentication_.
This alternative to just _authorization_ uses less overhead than
SPF/Sender-ID.  In the case of SPF versus DomainKeys, the bad is the
enemy of the good, safe, and fair. : )

-Doug

---

From: wayne <[email-masked]>
Date: Wed, 08 Jun 2005 21:12:50 -0500

> assumes "server authorization" is equivalent to "sender authentication."

This has been discussed many times on the spf-discuss list.  The
general consensus is that "server authorization" is *NOT* equivalent
to "sender authentication" and that is why the SPF-classic I-D uses
authorization throughout.  

> records is unwise for the majority of domain owners.

The above language for "Pass" is actually somewhat less strict than
the responsibilities assinged in the draft-mengwong-spf-* drafts.

> is using a rather heavy hammer.

Indeed it is a rather heavy hammer.  No question about that.  If you
could loan me a time machine, I would be quite willing to go back to
the fall of 2003 when this was first set up.

Unfortunately, this kind of HELO checking has been in place for a year
and a half now, there are lots of people who have published SPF
records under their HELO domains, and there are lots of people who are
checking these HELO domain SPF records even when the MAIL FROM is not
null.  (i.e. SpamAssassin)

The draft-schlitt-spf-classic-* series of I-Ds is an attempt to
document what is, not what should or could be.

>                                  The need for checking the HELO would be

In theory, SPF HELO checking is far less than optimal, but in practice
it isn't that bad.  Since you should publish SPF records at your HELO
domain if you publish SPF records at your MAIL FROM domain, a lot of
people are going to be publishing them.  It is not clear to me that
any other system is enough better to motivate people to switch.

> expected in this case, wouldn't something like BATV be more effective in

Why would SPF failure be expected in this case?

There are several reasons why people like SPF, limiting blowback is
just one reason.  

> released version of Sender-ID?

I don't know, but I do think that it is important to let people know
about the dangers.

I think it is likely that developers will need to update their MARID
protocol implementations anyway.  Instead of using the marid-protocol
document, they are now using the spf-classic document, and these two
documents are not completely compatible.

> express the scope of the record.

Mark Lentczner was the one who convinced Meng to remove the scope=
modifier from the SPF spec in the fall of 2003.  Hindsight is 20/20.

>                                   Depreciate the version 1 record.  Both
> records.

I see no particular reason why SPFv1 records should be depreciated.
They are well defined for the MAIL FROM and HELO identities which they
were published for.

> when eventually adjudicated as unfair.  Accept this banal reality.

Yes, that is possible.

My admittedly limited data shows that almost no one is checking things
for the MARID protocol.  At this time, it doesn't seem to be a
problem.

-wayne

---

From: Mark <[email-masked]>
Date: Thu, 09 Jun 2005 06:57:17 GMT

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Douglas Otis
> Sent: woensdag 8 juni 2005 21:20
> To: MARID
> Subject: draft-schlitt-spf-classic-02.txt
> 
> 
> assumes "server authorization" is equivalent to "sender
> authentication." It would be like me making a declaration that the
> postal service is authorized to deliver my letters, where recipients
> are then claiming any letter received from the postal service bearing
> my name is authentically or genuinely from me.

Your argument is essentially correct. Which is why I lobbied,
successfully, to change the wording "can now proceed with confidence in
the identity" to "can now proceed with confidence in the legitimate use of
the identity" (by the connecting client).

There is still a gap left between being "authorized" and being able to say
the return address is "authentic". From the perspective of a reputation
service, however, things are not as bleak as you portray. Because "pass"
is not just saying this or that IP is authorized, but is also,
effectively, an expression of trust. Trust that the MTA you authorized
will, itself, prevent abuse of its services (when perpetrated by its own
customers). Otherwise you would not continue to trust that MTA. And such
expressed trust, if shamed, can be withdrawn. And, if not withdrawn over
time, the meaning of your expressed trust, with "pass", can be said to
increase accordingly.

So that is why we, in the SPF Council, after heated debate, at long last
agreed on a compromise. We would state, that, after a "pass", one could
only "proceed with confidence in the legitimate use of the identity" (and
nothing more); but, that in the case of reputation, receivers could take
this expressed trust in the MTA to mean, especially when seen over a
prolonged period (and reputation is typically built up over time anyway)
that the domain takes responsibility for the reputation it builds up.

All-in-all, I think the resulting wording for "pass" was as even-handed as
we could make it.

- Mark 
 
        System Administrator Asarian-host.org
 
---
"If you were supposed to understand it,
we wouldn't call it code." - FedEx

---

From: Douglas Otis <[email-masked]>
Date: Thu, 09 Jun 2005 12:53:40 -0700

On Wed, 2005-06-08 at 21:12 -0500, wayne wrote:
> Douglas Otis <[email-masked]> writes:
> 
> > 1.  Introduction
> > ...
> > ,----

This draft still insists that by providing authorization for the server,
the mailbox-domain owner is "responsible for having sent the message."
How can you arrive at this conclusion without considering
"authorization" equivalent to "authentication?"  How do you know the use
of the mailbox-domain is "legitimate?"  You are assuming exclusive use
of the mailbox-domain is assured by the email provider.  

This introductory paragraph must include a dire warning there will be
those that consider "authorization" equivalent to "authentication."  To
protect the reputation of your domain, establish service level
agreements with your provider that indemnifies you, as the domain owner,
from the provider allowing non-exclusive use of your mailbox-domain,
whether for the bounce-address or the purported responsible address.

Those, in control of millions of zombies, will have at their disposal a
matrix of all possible mailbox-domains that authorize the outbound
server, detected by issuing a single message.  Those servers shared by
many domains will be an exploit bonanza.  This false assumption, which
goes well beyond what may be safely deduced from server authorization,
place a majority of mailbox-domain owner's reputation in peril.  

> the responsibilities assigned in the draft-mengwong-spf-* drafts.

The results remain unchanged. "Considered responsible for sending the
message" represents an irresponsible conclusion.  What assumptions
permit this conclusion?  Is it the domain owner that should know better
than to trust their email provider?  Or is it the recipient that should
know better than trust the mailbox-domain to be legitimate?  The
statements in the introduction and what is implied by a Pass condition
will result in considerable harm.

> document what is, not what should or could be.

This is not entirely true.  This draft can still recommend deprecating
the use of version 1 records, and document how version 2 records ARE
being used.  

> > The need for checking the HELO would be to protect network resources
> > using a unified name reputation service. The potential for hundreds
> > of DNS queries to resolve something as simple as HELO means this
> > scheme has a rather basic flaw.
> 

The inordinate number of DNS lookups mandated by SPF to resolve a
sizeable address list, when just a single host is being sought, pose
serious indefensible risk.  Motivation to change this practice will
likely become a matter of necessity at some point.

> Why would SPF failure be expected in this case?

2.1.  The HELO Identity
...
,----
| Note that requirements for the domain presented in the EHLO or HELO 
| command are not always clear to the sending party, and SPF clients 
| must be prepared for the "HELO" identity to be malformed or an IP
| address literal.  At the time of this writing, many legitimate e-mails
| are delivered with invalid HELO domains.
`----

> There are several reasons why people like SPF, limiting blowback is

Wouldn't BATV provide superior blow-back (DSN notifications sent to a
spoofed bounce-address) protection?  Unlike SPF, BATV achieves results
when implemented unilaterally.  Just removing original message content
in the DSN would also deprecate spoofing the bounce-address as a
delivery ploy for spam or viruses.

> documents are not completely compatible.

When you say MARID, do you mean Sender-ID?  It is unrealistic to expect
Sender-ID will change.  There are real and inexcusable dangers created
by how the scope-less version 1 SPF record is interpreted.  To insist
that documented methods in the use of SPF records must exclude the
scoped version of the record is nonsensical.  This abstinence from scope
enables a Sender-ID reputation trap.

> modifier from the SPF spec in the fall of 2003.  Hindsight is 20/20.

The time lines of these decisions should not preclude becoming aware of
a problem created by a competing draft.  Once aware of the results of
this capricious dismissal of a previously considered essential element,
act to reintroduce scope.

You should understand that publishing SPF records are not free from
risk, even without errors.  Much of the risk occurs as a result of those
that mistake authorization for authentication.  The domain supported by
server authorization can not safely be assumed to have originated the
message.  While it could be possible to refuse messages that are not
authorized by an SPF record demanding strict compliance, this is really
not practical either.

The real use for SPF is to offer greater protection for bulk email
distributors by way of white-listing implemented by major providers.  I
would even recommend that there be a scope devised, such as admin, to
specially limit the SPF record to this use.  This would mean there are
NO assurances being made for any mailbox-domain anywhere within the
message.  The domain owner simply wants to be trusted to do the right
thing and does not want to risk any contained mailbox-domain becoming a
victim of authorization-reputation.

> > Depreciate the version 1 record.  Both SPF "new classic" and
> > Sender-ID may continue to utilize these older records.
> 

This version of the SPF record is also seen as applicable for the PRA.
For a brief period of time it was.  The draft that includes this scope
for this early record also shares a common author.  You are really in a
weak position to insist they are wrong and you are right.  You will not
win this argument and harm those that believe otherwise.

> for the MARID protocol.  At this time, it doesn't seem to be a

Once insidious applications creep in that base reputations upon PRA
authorizations, while using the Machiavellian user feedback, this will
become a very real problem.  I can imagine it now.  "We supply the
tools.  These tools don't give mailbox-domains a bad reputation, people
give mailbox-domains a bad reputation."

-Doug

---

From: wayne <[email-masked]>
Date: Thu, 09 Jun 2005 20:34:17 -0500

> will result in considerable harm.

Well, SPFv1 has been deployed by thousands of domain owners since
early 2004.  After a year and a half of experience, only a very few
people seem to think this risk is significant.  I'm not going to
debate this issue with you since we have done so in the past and I see
no productive result of debating it again.

> the use of version 1 records, and document how version 2 records ARE

The whole point of spf-classic-* is to document how SPFv1 is being
used.  From what I can tell, version 2 records *aren't* being used,
and therefore the MARID protocol *isn't* being used.

>> it isn't that bad.  [...]

Ok.  I disagree, but I'll grant you an "I told you so!" if it ever
happens.

> delivery ploy for spam or viruses.

You can use BATV if you want.  Personally, I think it is a poor
re-invention of the ABBS/SES wheel, but YMMV.

The point remains that limiting blowback is not the only reason people
like SPF.

> When you say MARID, do you mean Sender-ID?

Yes, when I say "the MARID protocol", I mean what was one called
Sender-ID on this list.  Andy checked with the powers that be within
the IETF and they said that another name should be chosen due to the
trademark conflicts with Sender ID.  I am simply trying to follow
that request.

I'm going to skip the rest of your message.  Again, these are old
issues that I don't agree with your assessment and I have better
things to do with my time than to repeat myself.

-wayne

---

From: Douglas Otis <[email-masked]>
Date: Fri, 10 Jun 2005 12:11:35 -0700

> no productive result of debating it again.

While you may have an opinion that risks of being mistakenly attributed
for abuse is small, you should still offer a warning.  Unfortunately
your Introduction and description of PASS contributes to this risk,
rather than providing requisite cautions.  I doubt those finding
themselves blocked will be comforted by claims their situation is not
significant, nor worthy of consideration.

Due to this lack of concern, their provider was not made aware of the
added requirement of assuring mailbox-domain exclusivity to guard domain
owner's reputations.  You should also be aware of programs being
announced that will increase the level of risk that continued spoofing
may cause.  I am not asking you to anticipate the actions of an abuser,
but rather read announcements of major vendors.

Consider where this is headed.  Unlike IP address reputations, domain
names may be treated on a more permanent basis, perhaps waiting for
evidence of ownership change.  A reputation based upon the domain name
also provides no alternatives.  Unlike a bad reputation for an IP
address, a bad reputation for a domain name, even for a brief period,
could be extremely damaging.  In addition, there is no practical method
for restitution, as these authorization/reputation systems will likely
be processing the SPF record's multi-level results using a
filtering-paradigm.  This strategy may route a large number of messages
into a never visited junk folder. : (

> and therefore the MARID protocol *isn't* being used.

Does this change you mind?
http://informationweek.com/story/showArticle.jhtml?articleID=164301210

While I am pleased to see the majority of major providers avoided
publishing either version of an SPF record, there is always the
possibility some provider will attempt to assert influence by way of
acceptance criteria, perhaps found in some setup menu.  Review AOL,
ReturnPath, DoubleClick, or Skylist for examples of spf2.0/pra records.
Hotmail only publishes v=spf1 records, by the way.  Consider the impact
the announced program will have.  It is irresponsible to ignore this.

While you may espouse some ideal for what is contained within
spf-classic-*, there have been extensive efforts to re-engineer SPF
within this document's many iterations.  Dropping version 2 records from
the start, redefining record limits, depreciating the wildcard, adding
zone-cuts, removing zone-cuts, changing HELO handling, and then changing
HELO handling again.  Documenting version 2 of the SPF record may
diminish an illusory claim of greater acceptance.  However, when
Sender-ID uses v=spf1 records, these claims reveal only wishful
thinking.

A claim of SPF versus Sender-ID acceptance would be possible, only by
allowing the scope of the record to be explicitly declared.  Once it
becomes understood that providers must assure mailbox-domain
exclusivity, and unless there is a means to ensure the PRA is not
assumed, a responsible provider has the option of licensing the PRA
algorithm or not publishing SPF records.

Perhaps I should recommend an spf2.0/admin white-listing record.  In my
view, the only safe use of SPF would be for white-listing, where this
should also have an explicit scope for this use as well.  Such a
white-listing scope may in effect say "Do not base my customer's
reputation upon any server authorization, as mailbox-domain exclusivity
is not assured."

> trademark conflicts with Sender ID.  I am simply trying to follow

I'll refer to the draft-lyon-senderid-core as Sender-ID to avoid
confusion.  I am not writing a spec using this term, nor making claims
of use or ownership.

-Doug

---

From: wayne <[email-masked]>
Date: Fri, 10 Jun 2005 15:16:57 -0500

> http://informationweek.com/story/showArticle.jhtml?articleID=164301210

Considering that I posted a link to a similar story on the spf-discuss
list when the MS PR went out, no, of course it doesn't change my mind.

Note that I when I said "from what I can tell ... the MARID protocol
*isn't* being used", I used the present tense, while that article
talks about the future.

> the announced program will have.  It is irresponsible to ignore this.

I have not ignored this.

You may have already seen this, but it looks to me like the MARID
protocol has a little ways to go before it accepted even as an
experiemental standard.  See:

https://datatracker.ietf.org/public/pidtracker.cgi?command=print_ballot&ballot_id=1573&filename=draft-lyon-senderid-core

While it is quite possible that Microsoft will release Exchange 2003
SP2 with the MARID protocol despite "the conflicting use of the spf1
records between this proposal and the SPF proposal is harmful to the
Internet."  Microsoft is not immune to PR.

> the start,

Version 2 records were never part of SPFv1, they were never discussed
in SPFv1 specifications, and so they were never dropped.

>            redefining record limits,

The record limits have existed in the libspf2 implementation (the one
I wrote) since Feb 2004, long before MARID was even opened.  It is my
understanding that some other SPF implementations have also
implemented these limits.  This is an existing practice, and a change
to the spec that I feel is needed in order to make sure that SPF can
not be used as an effective DoS amplifier.  Yes, I did studies of the
actual packets when I came up with these limits.  Unfortunately, that
study was done a year and a half ago, and don't have all the results
now, just the conclusion.

>                                      depreciating the wildcard,

I'm not sure what you mean here.  SPF doesn't use wildcard DNS
records. 

>                                                               , adding
> zone-cuts, removing zone-cuts,

Zone cuts were indeed added to the mengwong-spf-01 draft a over a year
ago, and they were indeed removed from the schlitt-spf-classic-02
draft because they were not widely implemented and they were not
popular with the folks on namedroppers.  Several other unused features
have been dropped from the schlitt-spf-classic-02 I-D.  I think this
is a good thing, YMMV.

>                                changing HELO handling, and then changing
> HELO handling again.

Since the Dec 2003 "frozen" spec of SPF, the HELO handling has been
changed from "when MAIL FROM is null", to "MAY be done always" in
mengwong-spf-01, to "RECOMMENDED to be done always" in
schlitt-spf-classic-02.  SPF implementations that follow the Dec 2003
spec are still compliant today.

>                       Documenting version 2 of the SPF record may

I'm not involved in the documenting of the spf2.0 records.

> reputation upon any server authorization, as mailbox-domain exclusivity

Feel free to write up an I-D saying so.  

-wayne

---

From: Douglas Otis <[email-masked]>
Date: Fri, 10 Jun 2005 16:57:03 -0700

On Fri, 2005-06-10 at 15:16 -0500, wayne wrote:
> experimental standard.  See:
> 

SPF defines DNS record handling and syntax, but oddly was not included
in the list of I-Ds considered.  It will fall upon spf-classic to
resolve remaining record scoping conflicts.  Perhaps the next time this
comes up, there will be umpteen million copies of Sender-ID distributed.
Surely you are not surprised by this?

Do not depend upon anyone understanding the "scope" issue, when even the
SPF draft gives this subject such short shrift.  The average person will
trust the system based upon misleading claims made in either draft,
without becoming aware of the factors that may place them in peril.
Explain the dependency on the provider making requisite exclusivity
assurances of determinate mailbox-domains.

As painful as this seems, plan on requesting the v=spf1 records be
removed and replaced with "spf2.0/mfrom".  (Perhaps you could see fit to
define "spf2.0/".)

> in SPFv1 specifications, and so they were never dropped.

These newer records preceded the spf-classic documentation effort, which
is what I meant by dropped at the start.  Do you remember why the newer
scoped record was created?  It seems an understandably hostile decision
to exclude documenting use of this newer record.  Had this scoped record
been defined initially in this series of drafts, it would have been more
widely employed.  You want Sender-ID to be unable to utilize existing
records, and don't want a method to define the scope.  This needs to
change.

-Doug