Subject: RE: SPF PASS (was: "If you believe that the SPF concept is fundam entally flawed, please subscribe at http://www.imc.org/ietf-mxcomp/")

From: "Sukhenko, Alex" <[email-masked]>
Date: Thu, 26 May 2005 07:47:44 -0400

Sorry if I am jumping on mid stream and missing the main point but SPF was
suppose to be the first step and then next would be reputation.

BTY I am very pleased with SenderBase (even catches Comcast zombies) I would
be curious to know if anyone else is?

I also noticed that some corp. at least one Kraft, is rejecting email if
your sending IP is not one of your MX IPs:

"v=spf1 mx ?all"

----------------------------------------
Alex Sukhenko
CTO
Exmplar
3900 Jermantown Road
Fairfax, VA 20330
703 273 2311 x232 - phone
703 654 6588 - fax

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
On Behalf Of Carl Hutzler
Sent: Thursday, May 26, 2005 7:21 AM
To: [email-masked]
Cc: [email-masked]
Subject: Re: SPF PASS (was: "If you believe that the SPF concept is
fundamentally flawed, please subscribe at http://www.imc.org/ietf-mxcomp/")

[email-masked] wrote:

>Carl Hutzler wrote:
> 
>  
>
>>Is this use of SPF flawed?
>>    
>>
>[...]
>  
>
>>If the [connecting IP] = [SPF record] then "trust it
>>more/whitelist"
>>    
>>
>
>It's perfectly possible for a spammer to get a PASS.  You
>wouldn't whitelist a spammer.  But it's impossible for a
>spammer to pretend to be me, he'd get a FAIL (in my case).
>
>Unless I'm this spammer of course.
>
>  
>

Actually, we DO WHITELIST SPAMMERS. I mean it happens. We don't want it 
to happen a lot, but it does. See we also monitor everyone on the WL 
very closely via volume, complaint, and bounce rates. So while a spammer 
could get onto the whitelist, they won't be for long. And a new spammer 
will have very low limits placed on how much they can send via a simple 
SPF=whitelist type method. Now if they prove once on the SPF=whitelist 
that they are a good sender, we would bump up their rate limits....or if 
they contact us to get "accredited" via our postmaster.aol.com webpage 
where you can simply ask to be on the whitelist, we might let them in 
with higher limits on day 1. We do this of course for well known 
organizations.

>>valuable reverse MX records which cover well over 95% of
>>the email traffic on the internet today.
>>    
>>
>
>Is that a guess ?  95% is a rather high number.
>
>  
>
OK, 85%. Is that better? Still beats the 80/20 rule easily. How much 
mail is not sent directly from the sending ISP to the destination ISP. 
For AOL it is a small number.

>>Perhaps SPF should be updated to have the above logic.
>>    
>>
>
>You can use it this way.  But whitelisting a PASS only
>because it's a PASS is no long term strategy:
>
>    "v=spf1 +exists:{ir}.comcast.blackholes.us -all" 
>
>  
>

See above. It actually IS very good for AOL because of all our other 
reputation based logic. Perhaps not every ISP can build this, but we do 
have one. Of course many 3rd party products are out there that do what 
we do...SenderBase (volume/bounces), SpamCop (complaints), Spamnet 
(complaints), etc.

>>Back to my day job :-)
>>    
>>
>
>Be careful with mail from these comcast IPs, bye, Frank
>
>
>  
>

-- 
Carl Hutzler
Director, Host Mail Development
America Online
[email-masked]
703.265.5521 work
703.915.6862 cell

---

From: Andrew Newton <[email-masked]>
Date: Thu, 26 May 2005 10:43:26 -0400

> For AOL it is a small number.

So why aren't more people publishing very assertive SPF records ending 
in -all and why aren't more people only accepting mail that has a PASS?

Like Carl, I use SPF but not in the manner prescribed by the SPF 
documents.  I think there would be fewer detractors of SPF if more 
emphasis were placed on the "Framework" portion of the spec.  That's 
not to say that the work currently underway by Wayne is unimportant.

-andy

---

From: "william(at)elan.net" <[email-masked]>
Date: Thu, 26 May 2005 08:34:20 -0700 (PDT)

> So why aren't more people publishing very assertive SPF records ending in 
> -all and why aren't more people only accepting mail that has a PASS?

85% of email traffic (percent counted based on single messages) may well
be < 50% of email senders (percent counted based on individual authors)
and may be < 10% of email sender domains (percent counted based on domains
of individual senders).

Its that last number that is likely to be crucial because the first 2 
numbers involve people user communicates with most often or who are on 
the same net and those are likely to already be on their whitelist (if 
they have it), but user does not expect mail from others to be rejected 
or otherwise such user would already be using whitelisting-only setup.

> Like Carl, I use SPF but not in the manner prescribed by the SPF documents.

Unless I'm mistaken, you use it exactly in the manner prescribed by the EHLO
part of the spec, you're just downgrade results from MAILFROM check.

As I said long ago, EHLO should be totally separate document with separate
recommendations about its use (like CSV) and separate recommendations about
setting up records (because records should be a lot more precise and consist
of one or couple of "a" instead of entire ip blocks and includes).

And for each scope like EHLO it should be up to receiver if it wants to
use just one scope or several in combination with optional document 
that is available describing guidelines for doing combined checks.

> I think there would be fewer detractors of SPF if more emphasis were placed 
> on the "Framework" portion of the spec.

Sure, I'd love to get started on the "Framework" (aka UnifiedSPF), if only
we did not have well known political factors distracting us from the work.

-- 
William Leibzon
Elan Networks
[email-masked]

---

From: Tony Finch <[email-masked]>
Date: Thu, 26 May 2005 17:14:17 +0100

On Thu, 26 May 2005, Carl Hutzler wrote:
>
> I wish there was a way to utilize the relatively large number of SPF records
> in a technology like CSV.

It seemed to me at one point that there was some promise that it would be
possible to write an SPF record which applied only to HELO or only to PRA
or only to the return path. However divergent SPF/SenderID implementations
and specifications have horribly muddied the waters and I'm not sure it's
possible to do this now and expect any kind of reliable interoperability.
Disaster.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
BISCAY: WEST 5 OR 6 BECOMING VARIABLE 3 OR 4. SHOWERS AT FIRST. MODERATE OR
GOOD.

---

From: "william(at)elan.net" <[email-masked]>
Date: Thu, 26 May 2005 09:20:42 -0700 (PDT)

> in a technology like CSV.

We could easily write EHLO guidelines for SPF record checking and
publishing as separate document, kind of like BCP. In fact I'll 
keep this in mind and bring it up on spf-discuss when things are
a little more calm from current spf-classic draft discussions.

----

BTW - while we're talking about it, here is an algorithm for you to test:

1. Check if EHLO name is delegated in DNS, if so go to #2, otherwise
    skip to the end with result of UNKNOWN
2. Check if EHLO name has corresponding "A" record and if so go to #3
    else go to #4
3. Check if "A" record points to the ip of SMTP client, if so result
    is PASS, smile and skip to the end. Otherwise proceed to #4
4. Check if EHLO name has an SPF record. If it does, go to #5,
    otherwise go to #7
5. Try to do SPF test against EHLO name and client ip.
     If result is SPF pass, take it as PASS, smile and skip to the end.
     If result is SPF fail, take it as FAIL and skip to the end
     If result is SPF softfail, go to #6
     If result is SPF neutral or something else, take it as UNKNOWN and
     skip to the end
6. Check if EHLO name has one or more MX record. If it has none, then
    result is FAIL, skip to the end. Otherwise result is UNKNOWN.
    [If it has an MX, its good indication the name is used as domain
     and not as a host - most hosts do not have assigned MX]
7. If EHLO name is not TLD then cut down the first component of the name
    (i.e. for "host.example.com", cut down "host" to make it "example.com").
    Take this new cut name as if it was original EHLO name and proceed to #4
8. THE END

-- 
William Leibzon
Elan Networks
[email-masked]

---

From: "william(at)elan.net" <[email-masked]>
Date: Thu, 26 May 2005 09:43:41 -0700 (PDT)

> possible to do this now and expect any kind of reliable interoperability.

At the end of MARID we started on the path towards unified spf and 
multiple scopes. I'd not have been surprised if HELO was next scope
introduced and while I recognize contributions made by CSV people and
hard work they have put in their documents, a common syntax is better
for all and in the end, I believe their work would have been adoptable
as "restricted profile" SPF record (using "a" in place of SRV) as 
documentation for HELO scope.

IESG did not do us (I mean everyone who is interest in better email
anti-spoofing and security technologies) any favors by closing down
MARID when we finally started getting somewhere even if we were behind
on the deadline, other WGs are years behind in their work and do not
get disbanded (or need I mention USEFOR perhaps...)

And not that it matters, but AOL did not help either by agreeing to
support SID with use of spf1 only records (which brought political crisis
and strong unhappiness in spf community and stalled further work on
unified spf) - instead you should have pushed for quickly having rfc
(even if its not 100% perfect) that documents current spf1 with all
further work continuing on "spf2" with more comprehensive per-scope 
documentation and usage guidelines.

Oh well, enough of history... Lets hope we can salvage at least some of 
what we had worked for in the future even if it will not happen quickly.

-- 
William Leibzon
Elan Networks
[email-masked]

---

From: wayne <[email-masked]>
Date: Thu, 26 May 2005 12:03:37 -0500

In <[email-masked]> Tony Finch <[email-masked]> writes:

> or only to the return path.

Yes, at one point there was the ability to limit SPF version 1 records
to only certain scopes, but the scope= modifier was removed in the
fall of 2003.  At the time, we really couldn't think of a reliable way
of applying SPF records to the 2822.From: header, and it was thought
that distinguishing between the 2821.MAILFROM and 2821.HELO identities
was not needed.

After widespread experimentation with SPF started in Dec of 2003, it
made changes to the SPF specification very hard.

>                             However divergent SPF/SenderID implementations

The goal of the latest SPF-classic (e.g original SPF) I-D is to
document what exists, not to try and create a new SPF protocol.  We
have worked very hard to try and keep as much of the semantics of the
draft-mengwong-spf-0[01] documents as practical.

It is true that the MARID I-Ds[*] define a newer protocol with updated
semantics that include things like the return of scopes to SPF.  These
I-Ds also call for a new version number to be used, which is good, but
then these I-Ds say that the new semantics should be applied to SPFv1
records, which is REALLY bad.  I have asked the IESG to block the
publication of the MARID I-Ds due to the harm that will cause to the
existing SPFv1 deployment.  So far, the IESG has not agreed with this
and the incompatible MARID I-Ds are going forward.

-wayne

[*] At one time, Andy asked the PTB whether the existing SenderID
    trademark would cause legal problems for the IETF/ISOC if we
    published I-Ds using that trademark.  Andy came back and said the
    risk is too high and that SenderID shouldn't be used in IETF RFCs.

    Until such time that I hear that the PTB of the IETF have changed
    their mind, I am trying to avoid the use of "SenderID" when
    referencing the MARID I-Ds.

---

From: John Levine <[email-masked]>
Date: 26 May 2005 17:08:38 -0000

>> in a technology like CSV.

Yeah.  Too bad that among SPF's many flaws is that it completely
confuses the HELO domain and the MAIL FROM domain (and on a bad day,
the domains in From:, Sender:, Resent-From:, and Resent-Sender:.)

>It seemed to me at one point that there was some promise that it
>would be possible to write an SPF record which applied only to HELO
>or only to PRA or only to the return path.

SPF 2.0 was supposed to let you specify which contexts each record is
supposed to apply to, but I agree that it'll never get any traction.

---

From: John Levine <[email-masked]>
Date: 26 May 2005 17:10:42 -0000

>a little more calm from current spf-classic draft discussions.

We could, but since you can't tell whether the list of addresses in
an SPF record is supposed to apply to the EHLO address or the MAIL FROM
address or one of the message header addresses, what's the point?

The domain names in my EHLOs is completely disjoint from the set in my
MAIL FROM and mail headers.  How is SPF going to handle that?

---

From: Tony Finch <[email-masked]>
Date: Thu, 26 May 2005 18:19:44 +0100

On Thu, 26 May 2005, wayne wrote:
>
> These I-Ds also call for a new version number to be used, which is good,
> but then these I-Ds say that the new semantics should be applied to
> SPFv1 records, which is REALLY bad.

I strongly agree. This makes it impossible to use SPF's baroqueness to
advantage, e.g. by using BATV with stunt DNS servers, and other clever
tricks.

Tony.

---

From: "Hector Santos" <[email-masked]>
Date: Thu, 26 May 2005 13:33:32 -0400

----- Original Message -----
From: "Carl Hutzler" <[email-masked]>
Sent: Thursday, May 26, 2005 11:39 AM
Subject: Re: SPF PASS

> I wish there was a way to utilize the relatively large number of SPF
> records in a technology like CSV.
>
> -Carl

The basic issue with all of this is not a specific methodology can work or
not work, but rather the basic fact that none of it can be enforced.

So even if all the kinks were solved in SPF, CSV, FUZZ, ABC or what have
you,  SMTP developers, such as yourself, myself and others will still be
stuck with still ACCEPTING the non-compliant senders and/or dealing with
legacy issues.

It doesn't make sense.   The only, and I mean only reason I implemented SPF
was two reasons:

1) AOL.COM announced support from a domain standpoint. AOL being one of the
greatest sources of spam (spoofed or otherwise),  we need something from AOL
to help all the other victim systems.  So once AOL.COM announce SPF support
by publishing a SPF domain,  we immediately started to implement SPF, and
guess what, the AOL announcement also made MICROSOFT jump and in my view,
was the reason MARID eventually started

2) SPF would be part of a suite of other methods, in particular ONE method
that has no dependency with "new" information being available - this is the
CBV, the call back verifier.

The CBV is 100% based on the idea that "ok, we don't have much to go on, but
at the VERY least,  from a SMTP BOUNCE compliancy standpoint,  the return
path must be valid.  It may be spoofed, it may be a zombie, but it must be
valid. It can not be junk."

So we explored the CBV. I am not saying it is the "answer" but what it
proved was one fundamental aspect of the problem we are trying to address:

         That indeed, the majority, by atleast 60-80% of all transactions
         are non-compliant from a SMTP standpoint.

Period. No ands, ifs or butts. And it should not be a SURPRISE to anyone
because it is the reason why we have the problem in the first place.

So what's missing from the process?

We need an anchor that exist today to address the backward compatibility
issue.   We don't really have a consensus on what uniquely identifies a MAIL
sender or machine that is part of a mail domain.

So we can debate, argue left and right,  we are still left with the same old
issue on how to address backward compatibility.

Until someone can address this,  all the proposals are for the most part
worthless.

This is the secret the spammers don't want us to discover and realize,
because until we do, they have no reason for them to change or adapt.

SPF can work if everyone was forced to use SUBMITTER like idea to address
the transitional domain/ip changes.

CSV can work if everyone was forced to use reliable client domain name.

etc, etc, but as long as it is not "REQUIRED"  or enforceable, it is all
worthless.

So Carl,  how can we enforce these proposals?  How do I enforce CSV support?

No,  I am not encouraging "POLICY" discussions. I am talking about how can
we automate the framework of a mixed hetergeneous network of systems?

I have a few ideas.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

---

From: wayne <[email-masked]>
Date: Thu, 26 May 2005 12:55:26 -0500

> advantage, e.g. by using BATV with stunt DNS servers, and other clever

If you agree that this is really bad, I encourage you to tell
[email-masked] about this.

I had placed a warning about these problems in the spf-classic I-D
submitted early this year, but someone took it upon themselves,
without informing either Meng nor me, to remove this warning.  In the
lastest SPF-classic I-D, the SPF community came up with what I think
is a nice paragraph on the subject, with an explict reference to just
such stunt DNS server as you describe.  (See sections 2.4 and 9.3.1.2)

-wayne

---

From: Terry Fielder <[email-masked]>
Date: Thu, 26 May 2005 14:03:45 -0400

John Levine wrote:
> confuses the HELO domain and the MAIL FROM domain 
No it does not.  It checks each independently.  See my previous post to 
your question.

> (and on a bad day,

SPF never does that, please do not confuse PRA with SPF.

> supposed to apply to, but I agree that it'll never get any traction.

SPF 2.0 won't, because its tied to PRA.  That does not preclude SPF 3.0 
using the syntax of proposed SPF2.0, which would superscede SPF1 and 
really be the second generation of SPF1 (because SPF2.0 was *not* second 
generation of SPF, it was hijacking to a completely different usage and 
meaning (PRA))

-- 
Terry Fielder
[email-masked]
Associate Director Software Development and Deployment
Great Gulf Homes / Ashton Woods Homes
Fax: (416) 441-9085

---

From: wayne <[email-masked]>
Date: Thu, 26 May 2005 13:03:54 -0500

> address or one of the message header addresses, what's the point?

While far from perfect, it is possible to check the local part to see
if it is "postmaster" and use that to distinguish between HELO and
MAIL FROM.  In order for this to strictly work, you can't send email
using postmaster from hosts that you have these kinds of SPF checks.

So, you could say:

example.com            TXT "v=spf1 redirect=%{l}._spf.%{d}"
postmaster.example.com TXT "v=spf1 ... HELO policy ..."
*.example.com          TXT "v=spf1 ... MAIL FROM policy ..."

Yes, this very ugly.  Yes, if the designers of SPF knew two years ago
what we know today, we would have done a lot of things differently.
Lacking a time machine, we are kind of stuck with the semantics of
SPFv1 records.

-wayne

---

From: Terry Fielder <[email-masked]>
Date: Thu, 26 May 2005 13:59:33 -0400

> address or one of the message header addresses, what's the point?

Read on to see the point.

> MAIL FROM and mail headers.  How is SPF going to handle that?
I believe you will find that the EHLO/HELO is only checked if the MAIL 
FROM fails.  I think I have heard of other implementations where:
If the EHLO fails, you check the MAIL FROM, and if that passes then it 
gets an SPF pass.

Either way, it doesn't matter if your EHLO fails (it does on many, maybe 
even MOST systems), because as long as your MAIL FROM is SPF PASS, then 
your mails SPF response on the whole is a PASS, and is not negatively 
affected.

PS  This is from memory, one of the developers are better qualified to 
answer this question, if you don't trust my answer, but the 
generalization is at least correct, AFAICT.

When it comes time to using the authorization to compare to domain 
black/white listing, then the MAIL FROM domain and the EHLO/HELO domain 
could be used as a query to the lists.  So THEN a bad reputation of 
either the domain in your MAIL FROM or your EHLO/HELO could give you a 
bad score later down the line in SA or the like.  But I don't think 
anyone is doing that, if for no other reason then because there are no 
ways of confirming the email is from the domain it claims until SPF is 
deployed.

Terry

---

From: "John L" <[email-masked]>
Date: 26 May 2005 14:28:04 -0400

> No it does not.  It checks each independently.  See my previous post to your 
> question.

Could you show me the SPF records I would use to indicate that 
mta.example,com is valid as an EHLO but not as a bounce address domain 
while example.com is a valid bounce address domain but not an EHLO.  If 
it'll help, assume they both have an A record of 12.34.56.78.

> SPF never does that, please do not confuse PRA with SPF.

In real life, lots of people use SPF records for PRA, and there was a loud 
debate before coming to a consensus that the alternative of throwing 
everything away and starting over was unworkable.  I sure wish the SPF 
crowd had a better institutional memory.

R's,
John

---

From: Julian Mehnle <[email-masked]>
Date: Thu, 26 May 2005 20:30:34 +0200

> confuses the HELO domain and the MAIL FROM domain

I agree that the combined HELO+MAILFROM semantics somewhat limit the 
expressiveness of SPF, but as those semantics have been clear since the 
earliest days of SPFv1, I don't think this is a significant problem.

> (and on a bad day, the domains in From:, Sender:, Resent-From:, and
> Resent-Sender:.) 

SPF doesn't do that.  Sender-ID does.  Complain to Microsoft and the IESG.

---

From: Julian Mehnle <[email-masked]>
Date: Thu, 26 May 2005 20:36:44 +0200

> address or one of the message header addresses, what's the point?

If it starts with "v=spf1", its semantics is defined in draft-schlitt-spf- 
classic, which, in a long tradition, defines such records to apply both to 
the HELO/EHLO and MAIL FROM identites, and no others.

> MAIL FROM and mail headers.  How is SPF going to handle that?

Great!  If you are using separate sets of domains for HELO/EHLO and MAIL 
FROM, you can use separate SPF records for each of them.  Problem solved.

---

From: Terry Fielder <[email-masked]>
Date: Thu, 26 May 2005 14:40:26 -0400

John L wrote:
>> No it does not.  It checks each independently.  See my previous post 
>> to your question.
> 
> 

You cannot with SPFv1 (based on your assumption).  You missed the point: 
  It doesn't matter, primarily the HELO is only checked if the MAIL FROM 
fails.

A pass from the HELO or MAIL FROM results in SPF PASS status.

> In real life, lots of people use SPF records for PRA, 
Name one ISP MSP or ISP.  Small time single domain self maintained 
organizational mail servers don't really count in the volume of things, 
but just for the hell of it, try naming a few of those.

> and there was a 
> loud debate before coming to a consensus that the alternative of 
> throwing everything away and starting over was unworkable.  I sure wish 
> the SPF crowd had a better institutional memory.

I fail to see how the crowds memory has changed.  Currently the SPF 
council is writing the official "what is SPFv1" so then it can be 
carried forth to "what should be in the successor version".

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 26 May 2005 20:27:54 +0200

Carl Hutzler wrote:

> I wish there was a way to utilize the relatively large number
> of SPF records in a technology like CSV.

You could try the SIQ-idea:  input IP and FQDN, output PASS,
FAIL, or DUNNO.  The SIQ-server checks CSV and if n/a SPF to
create its (IP, FQDN, result, TTL) HELO-tuples.

If the SIQ-server is forced to use SPF (no CSV) to create its
records it could ignore all policies with the vague qualifiers
? or ~, and it could also ignore all "expensive" SPF policies,
anything with mx, ptr, include could be defined as "forget it,
too expensive for a simple HELO test".

So you only look at +/-a +/-ip4, +/-ip6, +/-all.  I'm not sure
about "mx" and "exists", a dedicated SIQ-server could have
enough time to evaluate these mechanisms.  Bye, Frank

P.S., you better add MTAMARK before CSV before restricted SPF

---

From: "william(at)elan.net" <[email-masked]>
Date: Thu, 26 May 2005 11:42:06 -0700 (PDT)

On Thu, 26 May 2005, Terry Fielder wrote:

> I believe you will find that the EHLO/HELO is only checked if the MAIL FROM 
> fails.

Why? Doing EHLO check is useful no matter if MAILFROM fails or passes!
And it does not make any sense to delay it until MAILFROM when EHLO is
the first identity parameter email server receives and this identity 
that has very few failure cases with SPF (unlike MAILFROM)

Also note that MAILFROM identity corresponds to original recipient 
(unless every forwarder starts using SRS and you change its semantics,
but that is unlikely) where as EHLO name corresponds to the mail system. 
As John proper said, they are disjoint and should not be mixed.

> I think I have heard of other implementations where:
> If the EHLO fails, you check the MAIL FROM, and if that passes then it gets 
> an SPF pass.

Unless I'm mistaken SPF pass is specific per scope/identity checked and 
there is no overall SPF pass.

> Either way, it doesn't matter if your EHLO fails (it does on many, maybe even 
> MOST systems), because as long as your MAIL FROM is SPF PASS, then your mails 
> SPF response on the whole is a PASS, and is not negatively affected.
>
> PS  This is from memory, one of the developers are better qualified to answer 
> this question, if you don't trust my answer, but the generalization is at 
> least correct, AFAICT.
>
> black/white listing, then the MAIL FROM domain and the EHLO/HELO domain could 
> be used as a query to the lists.

You should not query different identities on the same reputation service,
I believe that is a possibility for unpredictable and incorrect results
and reputation systems must be setup for each specific identity in question.

----

BTW, for those who do not participate at spf-discuss and don't know -
I'm writing paper on email identities and how they are used in path and 
cryptographic authentication methods (i.e LMAP and MASS) for anti-spoofing
protection. This is very much on the topic of what has been discussed here 
and on ietf-smtp about good and bad points of each proposal.

You can find the latest public draft at:

http://www.elan.net/~william/emailsecurity/path_and_cryptographic_authentication-draft-limited-07.htm

Comments on the content, especially if I got anything wrong are appreciated.

-- 
William Leibzon
Elan Networks
[email-masked]

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 26 May 2005 20:46:33 +0200

John Levine wrote:

> The domain names in my EHLOs is completely disjoint from the
> set in my MAIL FROM and mail headers.  How is SPF going to
> handle that?

Different FQDNs have different policies (for a HELO FQDN it's
generally possible to arrange things so that it's different
from MAIL FROM FQDNs)

That's exactly what Carl wants, use a _simple_ policy for the
HELO FQDN.  Or convince him to test CSV, and use CSV.  Or do
both, then he can do whatever pleases him.  For a HELO FQDN
"v=spf1 a -all" should be generally good enough.  Bye, Frank

---

From: Terry Fielder <[email-masked]>
Date: Thu, 26 May 2005 14:51:27 -0400

william(at)elan.net wrote:
>> FROM fails.

Right, I fell into the the JL trap of thinking that the two domains had 
to be the same and yet represent something different.  But if its a 
relay server, the HELO *will* be different and consequently *will* use a 
different SPF record (if one exists).

> there is no overall SPF pass.

That was not my understanding, but I accept being corrected if that's 
the case.

> 
>> Either way, it doesn't matter if your EHLO fails (it does on many, 
>> maybe even MOST systems), because as long as your MAIL FROM is SPF 
>> PASS, then your mails SPF response on the whole is a PASS, and is not 
>> negatively affected.
>>
>> black/white listing, then the MAIL FROM domain and the EHLO/HELO 
>> domain could be used as a query to the lists.
> 
> 

I disagree, a domain may authorize RELAY.com to relay his email.  But if 
RELAY.com is notoroious for sending spam (from his other customers he 
relays for), then when I check the HELO name I may want to reject the 
mail from RELAY.com because of his bad spamming reputation even if the 
"MAIL FROM" domain is SPF PASS for said relay.

Consider the case when RELAY.com is actually SPAMMER.com, and once 
SPAMMER.com is black listed you want to keep blocking mail from him even 
if he registers other throwaway domains to pretend his server 
SPAMMER.com is relaying for.  A thumbs up on the reputation of 
THROWAWAY.com may want to be ignored if a thumbs down occurs for the MTA 
SPAMMER.com

> protection. This is very much on the topic of what has been discussed 
> here and on ietf-smtp about good and bad points of each proposal.
>

---

From: "John L" <[email-masked]>
Date: 26 May 2005 14:56:13 -0400

>> mta.example.com is valid as an EHLO but not as a bounce address domain 
> You cannot with SPFv1 (based on your assumption).  You missed the point:  It 
> doesn't matter, primarily the HELO is only checked if the MAIL FROM fails.
>

My point, which I would have thought was obvious, is that SPF provides no 
way to say that EHLO example.com or MAIL FROM:<[email-masked]> are 
invalid.  In practice, I see quite a lot of forged mail like that, and 
SPF's inability to deal with it is a significant problem.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://iecc.com/johnl, Mayor
"I dropped the toothpaste", said Tom, crestfallenly.

---

From: Mark <[email-masked]>
Date: Thu, 26 May 2005 18:56:23 GMT

> From: [email-masked]
> [mailto:[email-masked]] On Behalf Of John Levine
> Sent: donderdag 26 mei 2005 19:12
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: SPF PASS (was: "If you believe that the SPF
> concept is fundam entally flawed, please subscribe at
> http://www.imc.org/ietf-mxcomp/")
>
>
>

Typically, HELO/EHLO is only used in the special case of MAIL FROM: <>, so
to have the SPF client check against postmaster@HELO. Since there is no
MAIL FROM identity to protect in case of an empty envelope-from, it would
suffice to just publish this simple SPF record (BIND style) for your
completely disjoint HELO/EHLO name:

completelydisjoint.mydomain.com.          TXT "v=spf1 a -all"

That will ensure, that in the case of a DSN, your own mailer is always
authorized (and no one else!). That easy.

- Mark 
 
        System Administrator Asarian-host.org
 
---
"If you were supposed to understand it,
we wouldn't call it code." - FedEx

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 26 May 2005 20:59:49 +0200

william(at)elan.net wrote:

> need I mention USEFOR perhaps...

For his sins (3710, MARID) Harald was now condemned to be a
co-chair, he has already found the first trap and pitfall ;-)

> Oh well, enough of history... Lets hope we can salvage at
> least some of what we had worked for in the future even
> if it will not happen quickly.

ACK.  Delete one erroneous SHOULD in senderid-core and move
on, I like this SIQ idea.  Bye, Frank

---

From: Terry Fielder <[email-masked]>
Date: Thu, 26 May 2005 15:13:39 -0400

wayne wrote:
> In <[email-masked]> "John L" <[email-masked]> writes:
> 
> 
>>while example.com is a valid bounce address domain but not an EHLO.
>>If it'll help, assume they both have an A record of 12.34.56.78.
> 
> 
> I'll start off by not answering your quesiton. ;-)
> 
> Personally, I would recommend just publishing these SPF records:
> 
> example.com       TXT  "v=spf a -all"
> mta.example.com   TXT  "v=spf a -all"
> 
> If you trust the host 12.34.56.78 enough to authorize it to use both
> the example.com and mta.example.com domain names, why wouldn't you
> trust it enough to use them in the right context?
> 
> 
> Ok, now I'll actually answer your question:
> 
> example.com                  TXT  "v=spf1 redirect=%{i}._spf.%{d}"
> postmaster._spf.example.com  TXT  "v=spf1 -all"
> *._spf.example.com           TXT  "v=spf1 a -all"
> 
> 
> 
> mta.example.com                  TXT  "v=spf1 redirect=%{i}._spf.%{d}"
> postmaster._spf.mta.example.com  TXT  "v=spf1 a -all"
> *._spf.mta.example.com           TXT  "v=spf1 -all"
> 
> 
> ugly as sin, and it means that you can't send email using the
> postmaster local part, but other wise it works and is well defined
> back to Dec 2003 (or earlier).
> 

I stand corrected, it can be done with SPFv1, hats off to Wayne (and I 
am going to keep this one for myself to use  :)

BTW  Ugly as sin is irrelevant, once setup only computers need to read 
it, and it can be as ugly and cryptic as necessary, as long as *it works*.

Terry

---

From: Julian Mehnle <[email-masked]>
Date: Thu, 26 May 2005 21:19:22 +0200

> while example.com is a valid bounce address domain but not an EHLO.

You can't.  The expressiveness of SPFv1 is limited in this regard.

However, SPF still effectively prevents forgery of both the "mta.example. 
com" and "example.com" identities, because no MTAs except those authorized 
by your SPF record(s) can use them, and those MTAs authorized by you 
should be under your control.

---

From: Julian Mehnle <[email-masked]>
Date: Thu, 26 May 2005 21:31:27 +0200

Terry Fielder wrote:
> William Leibzon wrote:
> > You should not query different identities on the same reputation
> > service, I believe that is a possibility for unpredictable and
> > incorrect results and reputation systems must be setup for each
> > specific identity in question.

William is right.  You should not confuse "example.com-the-HELO-domain" 
with "example.com-the-MAILFROM-domain".

You should not query "mailsenders.reputation-provider.com" for HELO 
identities or "mtas.reputation-provider.com" for MAIL FROM identities.

> "MAIL FROM" domain is SPF PASS for said relay.

Well, this scenario is actually one of the strengths of SPF (and other 
sender domain authentication methods).  Even if relay.com sends a lot of 
spam, if you _do_ trust them to prevent cross-user forgery[1], you may be 
able to still successfully get your own legitimate mail through, because 
receivers don't necessarily have to block on the relay.com MTA IP address 
any more, but can assess reputation on a per-domain basis instead.

References:
 1.  
http://www.schlitt.net/spf/spf_classic/draft-schlitt-spf-classic-01.html#cross-user-forgery

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 26 May 2005 21:33:34 +0200

Terry Fielder wrote in mxcomp:

>> Unless I'm mistaken SPF pass is specific per scope/identity
>> checked and there is no overall SPF pass.

> That was not my understanding, but I accept being corrected
> if that's the case.

The only "overall" thing in the actual draft is a HELO FAIL.

Hm... or it used to be in -00, -01pre1, -01pre2, -01pre3,
-01pre4, -01pre5, but was dropped in -01pre6, -01pre7, -01.

Now that's a bug.  Wayne, please fix this back, it was no
nonsense, I don't recall any debate / resolution about it.

And there was no -01pre5 vs. 01pre6 diff, and that's how I
missed it.

Not talking about HELO FAIL doesn't solve the pending HELO
PermError issue.

Without an overall HELO FAIL stuff like op=helo needs to be
checked, e.g. do we _need_ it ?  It's almost what Carl wants:

http://purl.net/xyzzy/home/test/draft-spf-6-3-options-06.txt

                      Bye, Frank

---

From: wayne <[email-masked]>
Date: Thu, 26 May 2005 14:49:34 -0500

In <[email-masked]> Terry Fielder <[email-masked]> writes:

>> [example snipped]

I first learned this trick from Greg Connor.  I'm not sure if others
realized it could be done earlier or independantly.

> it, and it can be as ugly and cryptic as necessary, as long as *it

Yeah, but I still don't see the point.  Configure your MTA to use the
right HELO domain, and configure your MTA to reject email claiming to
be from the MTA's host name, and you can just use normal SPF records.
Spammers can't abuse either name because they can't send email from
the right IP addresses.

-wayne

---

From: Terry Fielder <[email-masked]>
Date: Thu, 26 May 2005 15:04:51 -0400

John L wrote:
>>> mta.example.com is valid as an EHLO but not as a bounce address 
>>> domain while example.com is a valid bounce address domain but not an 
>>> EHLO.  If it'll help, assume they both have an A record of 12.34.56.78.
>>
>>
>> You cannot with SPFv1 (based on your assumption).  You missed the 
>> point:  It doesn't matter, primarily the HELO is only checked if the 
>> MAIL FROM fails.
>>
> My point, which I would have thought was obvious, is that SPF provides 
> no way to say that EHLO example.com or MAIL FROM:<[email-masked]> 
> are invalid.  In practice, I see quite a lot of forged mail like that, 
> and SPF's inability to deal with it is a significant problem.

Yes, and if people could write the final version the first time then, 
well, they wouldn't need versions, would they.

It's not that you don't want SPF, you just want it to do more.  Stay on 
board, there are good things coming once they get past the SPFv1 spec.

Terry

---

From: wayne <[email-masked]>
Date: Thu, 26 May 2005 14:03:45 -0500

-wayne

---

From: Carl Hutzler <[email-masked]>
Date: Thu, 26 May 2005 11:39:59 -0400

-Carl

---

From: Carl Hutzler <[email-masked]>
Date: Thu, 26 May 2005 16:07:39 -0400

>we automate the framework of a mixed hetergeneous network of systems?
So if AOL began doing CSV, then everyone would follow? Hmmm, it might 
help, but not sure that's a given.

---

From: wayne <[email-masked]>
Date: Thu, 26 May 2005 15:14:53 -0500

> nonsense, I don't recall any debate / resolution about it.

There was debate about it.  It is mentioned in the release notes.

What happened is that, as part of restoring the HELO checking from
draft-mengwong-spf-0[01], I copied that sentence into
draft-schlitt-spf-classic-00.  However, that sentence was originally
part of a "RECOMMENDED algorithm", so copying that sentence caused a
change in semantics.  The whole "RECOMMENDED algoright" has been
dropped.  It is Receiver Policy anyway, not Sender Policy.

-wayne

---

From: "Scott Kitterman" <[email-masked]>
Date: Thu, 26 May 2005 16:29:23 -0400

>[mailto:[email-masked]]On Behalf Of Carl Hutzler
>Sent: Thursday, May 26, 2005 11:40 AM
>To: [email-masked]
>Cc: Andrew Newton; [email-masked]
>fundam entally flawed, please subscribe at
What would be the advantage, from your perspective, to doing that instead of
using the HELO checking approach already defined in SPF?

Scott Kitterman

---

From: Carl Hutzler <[email-masked]>
Date: Thu, 26 May 2005 16:34:38 -0400

>using the HELO checking approach already defined in SPF?
Never even knew there was helo checking in SPF.  I might have to do some 
reading. My bad!

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 26 May 2005 23:05:10 +0200

wayne wrote:

> There was debate about it.

MID ?  Any _comma_ changed with the results is important.  It's
only luck that I tried grep 'overall' while writing a reply to
Terry / William.

> It is mentioned in the release notes.

| Removed e-mail receiver policy definition on how to handle
| HELO checking.  It was copied incorrectly from
| draft-mengwong-spf-01, changing its meaning.

Yes, obviously I didn't get the meaning of this part by only
reading the diff.

> It is Receiver Policy anyway, not Sender Policy.

Curious senders might wish to kow what the expected effect of
a HELO FAIL is.  Curious implementors might also wish to know
what to do in this case.  And if curious admins like Terry
consider to test MAIL FROM before HELO they waste time if it
is a HELO FAIL.

No receiver policy in the spec. is fine, but please don't get
carried away.  For a "who knows what the effect might be" text
100 KB are rather long, and there are already good PRG RfCs
not using DNS at all.
                        Bye, Frank

<http://mid.gmane.org/[email-masked]>
<http://mid.gmane.org/[email-masked]>

---

From: "John Glube" <[email-masked]>
Date: Thu, 26 May 2005 17:30:42 -0400

|-----Original Message-----
|From: [email-masked]
[mailto:[email-masked]] On Behalf Of Carl
Hutzler
|Sent: May 26, 2005 4:08 PM
|To: [email-masked]
|Cc: [email-masked]; Andrew Newton; [email-masked]
|Subject: Re: Addressing Backward Compatibility [was Re: SPF
|PASS]
|
|<snip>
|
|So if AOL began doing CSV, then everyone would follow?
|Hmmm, it might help, but not sure that's a given.

Possibly. However, if AOL were to say those senders who are
white listed, or who are applying to be white listed with
AOL something like:

As part of its ongoing efforts to improve network security
and its user's experience, AOL is implementing Certified
Server Validation (CSV). We urge all bulk mailers and
internet access services who are presently white listed
with AOL or who are seeking white listing status with AOL
to review this protocol and take the appropriate
implementation steps. For more details:

<http://www.mipassoc.org/csv/index.html>

At the same time, let it be known within the ISP and
marketing industries:

* We are implementing Certified Server Validation (CSV).
We encourage all bulk mailers and internet access services
sending mail to AOL to also implement CSV as appropriate.

Presuming you are satisfied that CSV does what you want:

* Set up a 'special' white list for those senders who have
published CSV records and indicate you plan to merge this
with your 'special' white list for those senders who have
published SPF records.

I believe that would get the ball rolling. As the saying goes,
"your wish is my command" :-)

John

John Glube
Toronto, Canada