Subject: Re: [Asrg] where the message originated (was: DKIM role?) (SM)

From: Ian Eiloart <[email-masked]>
Date: Fri, 16 Jan 2009 11:45:33 +0000

--On 12 January 2009 12:58:28 -0500 der Mouse <[email-masked]> 
wrote:

>> However, I do think that it's time that domain owners started taking
>> responsibility for the use of their domains.
>
> If I, say, were to forge mail from your domain to, say, some ibm.com
> user, would you take responsibility for it?  That's what you seem to me
> to be saying.

Not quite, but not far off. I'm not taking responsibility for your decision 
to forge the domain. However, I AM responsible for the fact that IBM have 
no easy way to detect that forgery. I SHOULD accept that IBM are therefore 
more likely to bounce that email back to me.

    When a spammer forges my domain, and the recipient bounces the message,
    they're doing what SMTP is designed to do. I shouldn't get pissed off 
with
    the recipient, if I've failed to publish records that allow them to
    check for forgeries.

What I should take responsibility for is publishing records that let IBM 
easily check whether the email is legitimate. For example, if I publish SPF 
records, and sign all my outgoing mail with DKIM, and require my users to 
use my Message Submission Service, then I think I've discharged that 
responsibility.

Having taken those actions (actually, I haven't but this is hypthetical), 
I'm quite happy to accept any bounce messages from IBM - provided they've 
checked for an SPF or DKIM record match before accepting the message.

Now, given that I haven't yet published SPF or DKIM records, I still get 
lots of spam bounced into my domain. Who do I blame? Well, in part it's my 
responsibility for failing to help the spam recipients. If I did publish 
those records, then recipient domains could more easily detect forgeries 
out of my domain.

SPF records would be useful because you can check them early, and cheaply. 
DKIM give senders the opportunity to get messages through forwarders.

Now, what I'm suggesting (not advocating yet, because I'm not certain that 
this is right), is that when there's no SPF record published, that we 
should not feel too bad about bouncing email because the domain 
administrator isn't taking adequate steps to protect the domain against 
spam blowback, and against phishing. Of course, I'm NOT suggesting that 
lack of an SPF record should score very high in any any spamicity measure, 
but it might count for something.

Now, an SPF or DKIM match gives us the huge gain that we can bounce 
messages selectively based on the content. Some recipients may not want 
certain message content, but by the time we've seen the content SMTP 
doesn't allow us to selectively reject. And, a pass allows us to 
confidently whitelist a domain or sender address without opening up a huge 
hole in our spam defences. For example, I've refused to whitelist UK 
academic research boards because they don't publish SPF records.

Now, the question is, what do we do when we see an SPF softfail, and no 
DKIM record? Well, at first I'd be reluctant to bounce the message, because 
I want to reward admins that are taking steps to protect their domains. 
However, that's not something that I'd be happy with on a permanent basis. 
Eventually, I want the world to be in a position where all forwarders check 
SPF on the way in, and rewrite sender domains. When we're sufficiently 
advanced in that project, then bouncing softfail would be more acceptable.

>
>> In a world where you *can* prevent abuse of your domain, what's wrong
>> with bouncing email?
>
> Depends on what that prevention requires.  ("Sure you can prevent abuse
> of your domain.  You just need to enter into a bilateral agreement with
> every receiving mailhost on the net.")

No, you just need to sign your outgoing email, and publish SPF records.

I know, someone's going to tell me that SPF breaks forwarding. Well email 
is already broken. I get up to a million spam messages into my domain 
daily. Everything we do to prevent spam causes problems ranging from 
additional load on our DNS servers to loss of important emails. The 
question is, are we going to fix it? I think what I'm proposing would at 
least allow people to do safe whitelisting

-- 
Ian Eiloart
IT Services, University of Sussex
x3148
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: SM <[email-masked]>
Date: Fri, 16 Jan 2009 10:00:17 -0800

At 03:45 16-01-2009, Ian Eiloart wrote:
>Now, what I'm suggesting (not advocating yet, because I'm not 
>certain that this is right), is that when there's no SPF record 
>published, that we should not feel too bad about bouncing email 
>because the domain administrator isn't taking adequate steps to 
>protect the domain against spam blowback, and against phishing. Of 
>course, I'm NOT suggesting that lack of an SPF record should score 
>very high in any any spamicity measure, but it might count for something.

What does SPF have to do with phishing?

I prefer to take adequate steps to prevent "invalid" bounces.  It 
helps me if the postmaster takes adequate steps for me to determine 
what is valid.  Some people do not use SPF due to its restrictions on 
email usage.

>messages selectively based on the content. Some recipients may not 
>want certain message content, but by the

DKIM-signed messages coming through this mailing list cannot be verified.

Regards,
-sm

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 19 Jan 2009 11:55:06 +0000

> What does SPF have to do with phishing?

Er, a lot. Would you rather bank with an organisation that published SPF 
records, or not? I know I would.

I'd also like an email client that tells me when the From: header domain 
doesn't match the return-path domain.

>
> I prefer to take adequate steps to prevent "invalid" bounces.  It helps
> me if the postmaster takes adequate steps for me to determine what is
> valid.  Some people do not use SPF due to its restrictions on email usage.
>

---

From: Dotzero <[email-masked]>
Date: Mon, 19 Jan 2009 09:07:16 -0500

>> DKIM-signed messages coming through this mailing list cannot be verified.

Ian,

While I appreciate your ardent fervor in support of SPF and DKIM, it
would appear that your practical experience is somewhat limited. For
the domains I'm responsible for I've sent 700 million+ DKIM signed
messages and in excess of 1 billion messages since changing the SPF
records for these domains to end with -all.

While I am supportive of both these approaches, I recognize that there
are specific ways that breakage occurs for otherwise legitimate mail.
There has been plenty of discussion on these issues on other lists
such as spf-discuss and ietf-dkim.

Your analysis of whether and how domains should implement these
approaches is somewhat simplistic and as you point out above, you
don't even eat the dog food that you advocate others should. An
exercise you might engage in.... what types of breakage occur when a
domain publishes -all at the end of their SPF record (Assuming that
receivers respect the published record and act accordingly)? What
sorts of breakage occurs for messages from a domain assuming they were
able to communicate that they sign all messages for a particular
domain?

Just a few thoughts.

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 19 Jan 2009 14:32:42 +0000

> such as spf-discuss and ietf-dkim.

Yes, I'm aware of the discussions. However, I'm also aware that email is 
already very broken in many ways. We seem to be stuck in a place where we 
can't move for fear of breaking something, even if the place we want to go 
would be so much better than where we are.

> don't even eat the dog food that you advocate others should.

Yes, I think I said that I'm engaged in discussing an idea, rather than 
advocating it. Probably I've actually said both things.

> An exercise you might engage in.... what types of breakage occur when a
> receivers respect the published record and act accordingly)?

That, I'm certainly not yet advocating. I'm suggesting that publication of 
records, even with "~all", is a huge step forward. It'll improve 
deliverability of properly submitted emails in domains that are able to 
maintain good reputation. That'll encourage deployment of proper MSA 
servers. It'll encourage deployment of MSA and DKIM compatible forwarding 
services, because they'll be more successful. Eventually, we'll end up in a 
place where more and more admins are happy to publish, and respect "-all" 
records.

Heck, I might even suggest that the UK (just cos that's where I am) should 
legislate that financial services organisations (those regulated by the 
Financial Services Authority) MUST publish SPF records and sign their 
outgoing email.

> What sorts of breakage occurs for messages from a domain assuming they 
were

---

From: Dotzero <[email-masked]>
Date: Mon, 19 Jan 2009 10:30:08 -0500

> would be so much better than where we are.

Note that I did not talk about breakage in general, I pointed out that
there are very specific ways in which mail breaks for each of these
approaches. I'm not speaking theoretically. As I pointed out, domains
I'm responsible for have sent a nontrivial amount of mail under both
of these approaches. Given that these domains have been heavily abused
we have accepted the tradeoff  between reduced abuse vs breakage of
otehrwise legitimate mail. This tradeoff may not make sense for other
domains. I reiterate, I am speaking as someone who as already
implemented.

> advocating it. Probably I've actually said both things.

Might I ask why you are not publishing SPF records that end in -all or
signing DKIM for all the email from your domains?

> maintain good reputation.

Why is ~all a huge step forward? What do you expect receivers to do
with mail from domains that make an assertion that translates to "I
don't really know where legitimate email purporting to be from my
domain eminates from"?

>That'll encourage deployment of proper MSA
> services, because they'll be more successful.

DKIM is not path based.

>Eventually, we'll end up in a

You appear to beleive that DKIM is dependent on SPF.

> Financial Services Authority) MUST publish SPF records and sign their

I might suggest that you check with the mail and abuse folks at
financial organizations and ask what issues they are encountering.

---

From: SM <[email-masked]>
Date: Mon, 19 Jan 2009 07:55:26 -0800

At 03:55 19-01-2009, Ian Eiloart wrote:
>Er, a lot. Would you rather bank with an organisation that published 
>SPF records, or not? I know I would.

I would prefer if my bank used a mechanism for email that protects 
the integrity of the content.

>I'd also like an email client that tells me when the From: header 
>domain doesn't match the return-path domain.

There are cases where the From: header may not match the Return-path 
domain; for example, messages from this mailing list.  It's also 
common for web-generated email traffic but that's mainly because most 
users are not aware that specifying the email address for the From: 
header doesn't set the email address for the Return-path.

The Return-path only tells us where to bounce the message.  It 
doesn't tell us who actually sent the message.  In paper 
communication, we use the signature and not the return address to 
validate authorship.  When we receive a letter, we rarely look at the 
return address.  That information is useful for the delivery 
service.  Most people don't determine where the message 
originated.  The only information we notice, if we have access to the 
envelope, is whether the letter is domestic or comes from a foreign country.

Regards,
-sm

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 19 Jan 2009 17:48:36 +0000

--On 19 January 2009 07:55:26 -0800 SM <[email-masked]> wrote:

> I would prefer if my bank used a mechanism for email that protects the
> integrity of the content.
>
> domain; for example, messages from this mailing list.  It's also common
> for web-generated email traffic but that's mainly because most users are
> not aware that specifying the email address for the From: header doesn't
> set the email address for the Return-path.

Yes, I know. Presumably your bank doesn't email you through this list or 
any other, though. What I'm after here is avoiding bank phishing. When 
communicating with my bank, I want them emailing me directly, to my current 
email address.

I'd be just as happy if they used DKIM to sign the message. I'd still need 
my mail client to tell me that it was signed properly, though. And, signed 
by the owner of the address that I can see in the message headers.

>
> The Return-path only tells us where to bounce the message.  It doesn't
> tell us who actually sent the message.  In paper communication, we use
> the signature and not the return address to validate authorship.  When we
> receive a letter, we rarely look at the return address.  That information
> is useful for the delivery service.  Most people don't determine where
> the message originated.  The only information we notice, if we have
> access to the envelope, is whether the letter is domestic or comes from a
> foreign country.
>
> Regards,
> -sm

---

From: Dotzero <[email-masked]>
Date: Mon, 19 Jan 2009 13:37:50 -0500

On Mon, Jan 19, 2009 at 12:48 PM, Ian Eiloart <[email-masked]> wrote:
> Yes, I know. Presumably your bank doesn't email you through this list or any
> other, though. What I'm after here is avoiding bank phishing. When

How happy will you be when your ISP checked the DKIM signature
upstream from your mail client, added in x-headers showing the check
(and possibly other things) and broke the signature?
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: "Chris Lewis" <[email-masked]>
Date: Mon, 19 Jan 2009 13:46:00 -0500

> (and possibly other things) and broke the signature?

Or, if your ISP checked SPF and DKIM, and reported the email as legit
... for a domain name very much like, but not exactly like your bank...?

Even with reputation services telling you whether the domain is
supposedly good, phishers generate domains so fast, and reputation
services are of necessity going to have to score new ones as "neutral"
at worst, it's not going to get you very far overall.

We need a feasible mechanism for forcing end users to be smarter ;-)
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: John Levine <[email-masked]>
Date: 19 Jan 2009 19:08:14 -0000

>>> domain doesn't match the return-path domain. ...

>any other, though. What I'm after here is avoiding bank phishing.

Have you actually looked at the mail from your bank?  As likely as
not, the return path points back at the ESP that sends their mail and
doesn't match the From: address.  Address matching is a totally
ineffective way to verify the source of any mail.  You also can't make
many assumptions about where it will be sent from; I got a real
message today from HSBC in the Channel Islands that was sent from Hong
Kong.

>by the owner of the address that I can see in the message headers.

Well, yes, we all know that effective use of DKIM requires some sort of
reputation system to decide what to make of the signature.

R's,
John

---

From: Dotzero <[email-masked]>
Date: Mon, 19 Jan 2009 14:04:39 -0500

> We need a feasible mechanism for forcing end users to be smarter ;-)

If you figure that one out Chris I'll provide your beverage of choice.
Personally I have come to the conclusion that once a questionable
email gets to the enduser (generically) all hope is lost. This is why
I have focused my efforts and thought around the MTA.
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: der Mouse <[email-masked]>
Date: Mon, 19 Jan 2009 14:44:24 -0500 (EST)

> Have you actually looked at the mail from your bank?

Thank you for explaining so clearly why I do not, ever, do anything
"online" with my bank.  (Makes it really easy to identify phishes. :)

Now, if they were to sign the mail with some content mechanism like PGP
or S/MIME, with the key available by going to the branch in person, I
might trust it.

More likely not, though, since I've seen enough over my bank's
personnel's shoulders to convince me they run Windows internally, so I
don't trust them to manage their private key data properly either.  (Of
course, if they got enough clue to start doing real signatures on
email, they might also grow a clue when it comes to managing their
internal systems.)

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		[email-masked]
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: Rich Kulawiec <[email-masked]>
Date: Mon, 19 Jan 2009 16:55:22 -0500

> integrity of the content.

That's an excellent point.  In addition, I would prefer my bank to
(a) not outsource their mail, (b) not send mail marked up with HTML
(the phisher's best friend) and (c) not send mail which includes any
URLs in the text.

( About (c): If they never send any, they can never typo them.  Nor can I,
when copying them from mail by hand or cut-and-pasting.  If I rely
solely on the single URL I entered -- very carefully, by hand, once --
then my chances of going to a typosquatted site drop considerably.
An attacker would need to gain control of the place I've stored that
URL, which would require gaining control of my computer, which would
mean that there would be no need for them to bother sending me a phish,
since they could just extract the URL/username/password triplet directly
the next time I used it.

Moreover, if the bank trained all their customers in this -- just like
they [try to] train them that they will never, ever ask for a password
-- then they'd be training their customers to be phish-resistant,
since they'd know that any message with a putative URL for the bank
is a phish.  This in turn would discourage phishers, who would be presented
with a reduced attack surface.  Maybe.  On a good day.  See Chris's comment
about educating users and recall Marcus Ranum's advice on that very topic:
"If it were going to work, it would have worked by now".  I concur that
we need to attack this problem at the MTA and network layers, because
by the time it gets to users, it's too late. )

But banks (and other financial institutions) don't do this.  It appears,
if I can attempt to intuit their priorities from the methods and content
of their email messages, that they are far more interested in marketing
and assessing marketing effectiveness than they are in message privacy,
security and integrity.  I think if it were otherwise, then, among
other things, PKI would have long since become widely deployed, and
they wouldn't actively be training their customers to click on links
in mail messages.

---Rsk

---

From: SM <[email-masked]>
Date: Mon, 19 Jan 2009 16:02:19 -0800

>(the phisher's best friend) and (c) not send mail which includes any

The economy and specialization works in favor of (a) and marketing in 
favor of (b).

>since they could just extract the URL/username/password triplet directly

We commonly see URLs in emails which are different from the site the 
actual link points to.  We live in an age of convenience.  People 
find it easier to click on a URL than to copy and paste or type in 
the URL.  It doesn't require a leap of imagination to see how all 
this contributes to phishing.

It's difficult to counter (c) as business communication is centered 
around how to get people to a web site as it's seen as a low-cost and 
accessible medium to interact with the customer.

>with a reduced attack surface.  Maybe.  On a good day.  See Chris's comment

Customers may have been trained not to give out their password if 
they are asked (over the phone) but they are giving it out by logging 
into an web site.  If banks were to implement (c), then it does, as 
you put it, reduce the attack surface.

>by the time it gets to users, it's too late. )

Responding to Chris' comment, there isn't a feasible mechanism for 
forcing end users to be smarter.  Utilization of a technology should 
not be an IQ test.

Don't read all this as saying that I don't agree with your 
points.  It's more about whether we are taking the right approach to 
solve the problem.  I agree that each time the problem gets to the 
users, it's a disaster in waiting.  One of the issues in dealing with 
the problem at the MTA level is that we have to second guess the 
user's desires and we can get it wrong.

>they wouldn't actively be training their customers to click on links

You hit the nail on the head there.

Regards,
-sm

---

From: Franck Martin <[email-masked]>
Date: Tue, 20 Jan 2009 12:27:25 +1200 (FJT)

Well, take a stroll and go to talk to your marketing manager, ask him what he thinks of email as a marketing tool and what you can do or cannot do with it.

You may be surprised.

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 20 Jan 2009 11:43:31 +0000

> ... for a domain name very much like, but not exactly like your bank...?

That would be a problem. However, a smart mail client could also check that 
the domain is used in my address book. And, it could also check my address 
book and history for matching domains. Near matches should be flagged as 
suspect.

Use of relatively secure top level domains could help. I don't understand 
why my bank stopped using its ".coop" domain. It's much harder for spammers 
to register those domains. I'd like to see a ".fin.uk" (or similar) domain, 
where only organisations regulated by the Financial Services Authority can 
register domains, and where they're required to use SPF.

> at worst, it's not going to get you very far overall.

Frankly, I'm not going to bank with a bank that can only achieve "neutral" 
for it's email domain reputation.

> We need a feasible mechanism for forcing end users to be smarter ;-)

Agreed! Banks that use email should take care to educate their users on 
this matter. Holding the bank (rather than the account holder) at least 
partially responsible for fraudulent use of accounts that they manage would 
help. They might have to prove that they'd taken reasonable steps to 
educate their customers about phishing, and that they'd deployed domain 
protection technologies.

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 20 Jan 2009 11:49:40 +0000

--On 19 January 2009 19:08:14 +0000 John Levine <[email-masked]> wrote:

> ineffective way to verify the source of any mail.

Right now, that's true. If the banks were serious about preventing 
phishing, they'd do something more sensible.

> You also can't make
>> I'd be just as happy if they used DKIM to sign the message. I'd still
>> need  my mail client to tell me that it was signed properly, though.
>> And, signed  by the owner of the address that I can see in the message
>> headers.
>

Absolutely. I should be able to tell my mail client or ESP which domains 
and addresses to trust (my bank, my employer, my doctor...), in the first 
instance. Only when DKIM and SPF are in wider use will a centralised 
reputation system be very useful. But "wider use" needn't mean most of the 
world. It could just mean, "most of the financial institutions in the UK", 
for it to be of utility to most people in the UK. Or "most academic 
institutions in the UK" for it to be of use here.

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 20 Jan 2009 11:55:02 +0000

> The economy and specialization works in favor of (a) and marketing in

Yes, but legislation requiring banks to do sensible things here, is 
feasible. At least, it is in the UK and probably elsewhere given their 
current reputation for incompetence.

Actually, it's not the outsourcing that's the problem. They just need to do 
that properly, with sensible return-paths and appropriate SPF records.

If banks were doing that properly, it would be easier for ESPs to detect 
phishing, then you could conceivably hold them responsible when they fail 
to do so.
-- 
Ian Eiloart
IT Services, University of Sussex
x3148
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: Rich Kulawiec <[email-masked]>
Date: Tue, 20 Jan 2009 07:10:15 -0500

On Tue, Jan 20, 2009 at 11:49:40AM +0000, Ian Eiloart wrote:
> Only when DKIM and SPF are in wider use will a centralised  
> reputation system be very useful. 

Don't hold your breath.  SPF is dead -- and good riddance, it was a very
stupid idea -- and a centralized reputation system of any kind would
be a disaster.  It's one of the very last things anyone who is actually
opposed to spam should ever want to see happen.

---Rsk

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 20 Jan 2009 13:40:59 +0000

> opposed to spam should ever want to see happen.

I disagree fundamentally about the centralise reputation system. My most 
effective anti-spam tool is a centralised reputation system for IP 
addresses - the two RBLs that I consult. For example, a few days ago, I 
rejected 260,000 messages and delivered 56,000. 170,000 were rejected as a 
result of RBL lookups. I get very few complaints about false positives from 
those RBLs.

Anyway, if you don't want to use a centralised reputation system, then use 
a local one. I would do that in the first instance, then fall back to a 
centralised system when I got a neutral result.

The key thing is that you can't use any kind of email domain or email 
address based reputation system unless you have evidence that the sender 
address isn't forged.

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 20 Jan 2009 13:44:36 +0000

--On 19 January 2009 13:37:50 -0500 Dotzero <[email-masked]> wrote:

>> communicating with my bank, I want them emailing me directly, to my
>> current email address.
>>
>> need my mail client to tell me that it was signed properly, though. And,
>> signed by the owner of the address that I can see in the message headers.
>>
>

That would make me unhappy. Presumably they're not supposed to do that, but 
doesn't DKIM allow the signer to say what they're signing? And, doesn't 
that survive addition of new headers? If my ESP (of course I don't use my 
ISP for email) broke a DKIM signature, I'd expect them to replace the 
signature with a good one. But, I'd prefer that they simply delivered the 
message unbroken.

Of course, all of this works better when it's correctly implemented.

---

From: Dotzero <[email-masked]>
Date: Tue, 20 Jan 2009 09:02:31 -0500

On Tue, Jan 20, 2009 at 8:44 AM, Ian Eiloart <[email-masked]> wrote:
> doesn't DKIM allow the signer to say what they're signing? And, doesn't that
> survive addition of new headers? If my ESP (of course I don't use my ISP for
> email) broke a DKIM signature, I'd expect them to replace the signature with
> a good one. But, I'd prefer that they simply delivered the message unbroken.
>

Why aren't they supposed to do that? How many MUAs are checking DKIM
signatures at the moment? I can think of a numbe rof ISPs that are
adding headers to indicate various checks. From a DKIM perspective,
the less you sign the greater the risk. Don't do the body length (full
length that is)? Evil bad person can engate in replay attacks with
content of their choice.

This is not only about what you prefer personally. This is about what
works generally. You work for a University. What percentage of your
endusers are checking DKIM? I'm going to take a wild leap and assert
that it approximates zero if it isn't actually zero.

As far as I know, DKIM checking is happening at the MTA or in
association with the MTA (Seperate host behind the front line MTA or
something similar). It isn't happening at the user level. And that
makes sense, at least to me. Maybe in some distant future MUAs will do
DKIM checking but I wouldn't hold my breath.
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: John Levine <[email-masked]>
Date: 20 Jan 2009 14:53:54 -0000

>current reputation for incompetence.

I suppose.  I sometimes drop by the weekly computer security seminar
at the Cambridge computer lab, where I've heard some amazing stuff to
buttress that reputation for incompetence.

>Actually, it's not the outsourcing that's the problem. They just need
>to do that properly, with sensible return-paths and appropriate SPF
>records.

Why do you keep harping on SPF?  Of all of the proposed security
schemes on offer, it is by far the worst.  It doesn't even attempt to
secure any part of the message visible to recipients, and it only
works for the subset of mail that is sent from a fixed point to
recipients who don't remail or forward it.  (Despite endless claims by
SPF's fans, that last part is a fundamental design failure of SPF, not
of the mail system.)

DKIM at least starts to address those problems, but it still doesn't
begin to try to deal with the much harder problem of lookalike
domains.  Let's say you get a message from [email-masked], which
is 100% DKIM, SPF, and Sender-ID approved.  Is that Paypal?  How can
you tell short of manually looking up WHOIS registrations?

R's,
John

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 20 Jan 2009 14:54:27 +0000

--On 20 January 2009 09:02:31 -0500 Dotzero <[email-masked]> wrote:

>
> Why aren't they supposed to do that?

Well, because they haven't delivered the message yet, for example. They 
might be about to forward it to another MTA. I know that Exim does most of 
its header manipulation before delivery.

> How many MUAs are checking DKIM
> signatures at the moment?

Probably none, but it's an obvious thing to want to do, isn't it? Anyway, 
what's the point of delivering a message with DKIM signature that you've 
already broken. At least, you could remove the signature.

> I can think of a numbe rof ISPs that are

---

From: Dotzero <[email-masked]>
Date: Tue, 20 Jan 2009 10:19:29 -0500

> its header manipulation before delivery.

You are assuming facts not in evidence. They might also be about to be
hit by an asteroid.

> already broken. At least, you could remove the signature.

Hmmm...I dunno..... maybe because the header they added says something
like..... "DKIM check....PASS". that is, they are creating something
that a human being can look at today and go....:"YEA!  my ISP or Mail
Admin cares enough about me to check DKIM AND to let me know what the
otucome of the check was." I could see some people going "Damn my ISP
or Mail Admin for breaking something today because at some distant
point in the future I or someone else might want to take advantage of
that". While the former might be a small number in relation to all
users, the latter is probably a number orders of magnitude smaller
again.
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: der Mouse <[email-masked]>
Date: Tue, 20 Jan 2009 10:23:32 -0500 (EST)

> Don't hold your breath.  SPF is dead --

Interesting you should say that.  Just yesterday I had to diagnose a
mail bounce for someone; it proved to be a sender who advertised SPF
with -all sending mail thorugh an alias expansion on my mailserver.
(The really discouraging part is, what did the sending domain's admin
do?  Drop SPF?  No; he added my machines to his SPF record!)

> and good riddance, it was a very stupid idea --

Actually, it was a good idea, at the core.  But the devil is in the
details, and the details chosen were ones that, um, don't work well.

> and a centralized reputation system of any kind would be a disaster.

Strongly disgaree.  There are almost as many centralized reputation
systems out there right now as there are DNSBLs (some DNSBLs, such as
noprimes, are not reputation systems), and the best of them are very
far indeed from disasters.  (Well, possibly excepting to those who want
Atlas to shrug in this case.)

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 20 Jan 2009 15:35:33 +0000

--On 20 January 2009 10:19:29 -0500 Dotzero <[email-masked]> wrote:

>> might be about to forward it to another MTA. I know that Exim does most
>> of its header manipulation before delivery.
>>
>

Maybe not so distant:
<http://mipassoc.org/pipermail/ietf-dkim/2008q1/009039.html>

This is a plea from Dave Crocker (author of rfc822) to take a Thunderbird 
extension that checks spf and dk, and add support for dkim.

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 20 Jan 2009 15:39:55 +0000

> do?  Drop SPF?  No; he added my machines to his SPF record!)

With a "~" or "?" I hope. That's reasonable given that he'd got notice from 
a contactable administrator.

---

From: John Leslie <[email-masked]>
Date: Tue, 20 Jan 2009 10:50:05 -0500

John Levine <[email-masked]> wrote:
> schemes on offer, it is by far the worst.

   SPF has been due for its 2-year review (of Experimental Status) for
a while now. Hopefully it will get a useful review, and I choose to hope
it will get enough of an overhaul to become useful. With some simple
changes it could give a measured likelihood that a 5321-MailFrom deserves
to be trusted for generating bounces -- a major win.

> It doesn't even attempt to secure any part of the message visible to
> recipients,

   Nor should it, IMHO. Other approaches are better for that.

> and it only works for the subset of mail that is sent from a fixed point
> to recipients who don't remail or forward it.

   This feature could be substantially mitigated in a number of ways.
Unfortunately the Experimental spec freezes a moment in time where MARID
was struggling with that issue (and others) without having time to reach
a useful consensus.

   Fundamentally, of course, the attempt to have one-size-fits-all
processing by the receiving MTA is dubious. It's not the coding of SPF
records that breaks forwarding: it's the processing of them. Relaxing
the processing rules could help a lot.

> (Despite endless claims by SPF's fans, that last part is a fundamental
> design failure of SPF, not of the mail system.)

   The original SPF design tried to be all things to all emailers --
a hopeless task. I think two years of experience in Experimental status
has moderated that desire. And I see promise in the use of the pending
Authentication-Results header (though I must agree with Doug Otis that
it would be stronger if it included the IP address).

> domains. 

   Nor should it, IMHO.

> Let's say you get a message from [email-masked], which is 100%
> DKIM, SPF, and Sender-ID approved. Is that Paypal? How can you tell
> short of manually looking up WHOIS registrations?

   Most folks couldn't tell if they _did_ look up WHOIS -- so at first
blush I'd say that's the wrong question.

   Let's think about it differently.

   Why does phishing work?

   It works because the security of financial transactions depends on
obviously insecure passwords (anything simple enough for average folks
to remember _must_ be insecure) entered onto loosely secured websites.

   Compare that to ssh. Is there a record kept of what certificate is
used? Are there obvious warnings when you start a session with a
server whose certificate you've never seen before? Or even a warning
when the certificate changes?

   More to the point: why do financial institutions depend upon code in
browsers instead of calling a separate application for authentication?
The quality of security in browsers varies from barely adequate to
downright laughable (with a lot of customers using outdated browsers
closer to the laughable end of that range).

   Is there actually any point in trying to solve phishing issues by
verifying the origin of email if the customer is going to depend on
a known-insecure web-browser?

--
John Leslie <[email-masked]>
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 20 Jan 2009 17:05:49 +0100

> do?  Drop SPF?  No; he added my machines to his SPF record!)

(He probably did so because you provide no name to include...)

Setting SPF records is error-prone, as there is no way to collect 
results of test carried out with the softfail (~) qualifier.

> details, and the details chosen were ones that, um, don't work well.

Yet, if the idea was good, it might be worth to rescue it. There is 
not such an abundance of good ideas at the transfer level to throw 
away the ones that don't work smoothly right from the start. And it is 
fair to note that, for some reasons, SPF is still in its experimental 
infancy.

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 20 Jan 2009 16:13:05 +0000

> opposed to spam should ever want to see happen.

So dead that of hotmail.com, gmail.com, mac.com, apple.com, microsoft.com, 
facebook.com, ebay.com only er..., all of them publish SPF records.

Hmm, lets take a look at the email I've received today. These are the top 
20 domains that have delivered to one of my cluster machines. Several of 
the domains do use -all in their records, but most don't.

Facebookmail.com top the list at 10% of inbound mail. They do publish spf 
records. Of the rest, three yahoo domains, and one other don't have spf 
records. One is ours, one is null (bounces), and the remaining 13 domains 
all do publish spf records.

The 14 domains in the top 20 account for 28% of all the email that we 
accepted for delivery. That's certainly enough to justify building a local 
whitelisting mechanism against.

---

From: der Mouse <[email-masked]>
Date: Tue, 20 Jan 2009 11:28:57 -0500 (EST)

>> (The really discouraging part is, what did the sending domain's
>> admin do?  Drop SPF?  No; he added my machines to his SPF record!)
> With a "~" or "?" I hope.

No; just s/-all/mx:rodents-montreal.org &/  (You might find this more
reasonable when you know that the admin in question and I have known
one another personally for years.)

But his competence at writing SPF records is not my real point.  My
point is that this is an example of a sender and a different receiver
who have SPF in live use today.  Misguided, perhaps; broken, perhaps -
but "dead" seems to me to be overstating the case.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 20 Jan 2009 16:35:11 +0000

>> short of manually looking up WHOIS registrations?

Well, without all those technologies, it's simple to simply use paypal's 
domain. Then there's no clue. Now, if you use a look-alike domain name, 
then you're probably violating the trademark. That's illegal, so your ESP 
and your mail client will be quite justified in looking for domains that 
are similar to ones that you trust. That list might come from several 
sources - trademark registrars, your address book, your whitelist, and so 
on.

> browsers instead of calling a separate application for authentication?

Because, when the security is breached the customer pays. That needs to 
change. Make the banks liable for frauds that are committed against them, 
and then they'll start taking it seriously. They'll block insecure 
browsers, and the browser authors will be forced to catch up.

The downside is that it's their poorest customers who may be forced to pay 
for hardware or o/s upgrades. There's also a risk that they might decide to 
only support one browser.

---

From: "Chris Lewis" <[email-masked]>
Date: Tue, 20 Jan 2009 12:03:49 -0500

> extension that checks spf and dk, and add support for dkim.

I remember that and the plea.

The plugin didn't do SPF.  It couldn't do SPF, because it didn't have
access to either the MAIL FROM or HELO/EHLO.  It did something similar
to SPF (From:).

Even to get it to do From: "properly" required intimate knowledge of
your mail server architecture.

You could make it do SPF properly if you hacked your MTAs to add various
headers showing details about the perimeter connection (HELO, IP, rDNS,
MAIL FROM).

We do insert such info, but it obviously wasn't going to be worthwhile
hacking the plugin to recognize our additional headers.

The DK stuff didn't work at all AFAIK.

It was an experiment.  As an experiment, I believe it died.
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: Dave CROCKER <[email-masked]>
Date: Tue, 20 Jan 2009 11:44:26 -0800

> you tell short of manually looking up WHOIS registrations?

Two points:

1.  The word "security" is not all that helpful in these discussions.  To 
security technology experts, it means too many things to know what is meant in a 
particular case.  To the rest of us, it carries too little precise meaning.

2.  Almost all of the discussions about these authentication-related 
technologies start with the authenticated identifier and then talk about what 
you can derive from it or how it might be spoofed or otherwise abused.  Since 
that is exactly what must be done with suspect messages, the model seems to make 
sense to (re-)apply for authentication.  I now believe it is exactly the wrong 
model.  That's a model that looks for the problem, the attack, the basis for 
suspicion.

A very different model starts as a model of trust and specifies a table of who 
is part of the model.  In other words, it works in the oppsoite direction. When 
a message shows up, the only question is whether that message falls under that 
umbrella of trust.  If it doesn't, then we are done with that model, for that 
message.  Any further consideration of that message has nothing at all to do 
with the trust model.  The authenticated identifier is the sole, simple lookup 
string into that the model.  Either the identifier string works or it doesn't.

Concerns about spoofing the identifier, re-purposing its use, and corrupting 
the content -- authentication, authorization and data and signature integrity -- 
are all valid for evaluating the underlying technology but they really are 
secondary to any serious discussion of the model.  After the briefest review to 
ensure that a particular technology has made specific and acceptable choices for 
the "security" questions, the questions should not burden debate about use of 
the technology.  Instead we seem to find these questions hashed, rehashed and 
hashed again, as the focus of community debate.

When the anti-abuse world discusses use of these authentication technologies, we 
need to stop being the anti-abuse world and start being the trust world.  The 
anti-abuse world really has the receiving assessor as an isolated island of 
suspicious.  In today's Internet, that's required.  But the trust world is quite 
different.  It is collaborative -- between signer and receiving assessor -- and 
deterministic, rather than vague and heuristic, and strictly the effort of the 
receiving assessor.

Totally different model.  So we need a totally different approach to talking 
about it.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: John Levine <[email-masked]>
Date: 21 Jan 2009 00:56:55 -0000

>then you're probably violating the trademark. That's illegal, ...

Uh, dude, we're talking about phishing here.  If that's not already
illegal in Australia, I think I've found a major recession-resistant
business opportunity.

R's,
John

>> a known-insecure web-browser?

Maybe.  One of my bank accounts requires me to use a physical dongle
to generate a code number.  I expect in the future they'll give you a
USB dongle with a small screen and a couple of buttons so you do most
of your banking session on the computer, but when you hit go, the
dongle lights up with the details of the transaction the bank is about
to do and you have to push YES or NO on the dongle to confirm.  That
seems like it could be made reasonably secure.
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: John Levine <[email-masked]>
Date: 21 Jan 2009 01:04:27 -0000

>> stupid idea --

For largely political reasons, Hotmail requires anyone sending a
significant amount of mail to them to have Sender-ID records.

For senders that are on its whitelist, AOL reverse engineers the IP
addresses to whitelist from the sender's SPF records, which is way
easier all around than the former mostly manual system.

Since S-ID falls back to SPF records, most senders just publish one set of
SPF records for both.  Note that neither of these are using SPF for its
nominal purpose; I'm not aware of any large system that does.

R's,
John

---

From: Rich Kulawiec <[email-masked]>
Date: Tue, 20 Jan 2009 21:07:48 -0500

> I disagree fundamentally about the centralise reputation system.

Every attempt made at these thus far has been an absolute failure.
(See "TrustE" and "Habeas", for example.)  I can't think of any
reason why the outcome will change if the experiment is run again.

BTW, I don't consider DNSBLs and RHSBLs to be reputation systems.
That's not a knock on them: on the contrary, they are, by an enormous
margin, the most effective anti-spam measures we have.  But they're
not trying to be reputation systems, and they don't need to be.

> address based reputation system unless you have evidence that the sender  

I disagree, because I don't really *care* whether mail is forged or
not, and regard that as largely an insignificant problem, when compared
to the monumental problem posed by spam.  (Morever, for most purposes, it
doesn't matter: when a known spammer domain shows up, reject the message.
If it was really from them: correct decision.  If it was from someone
dumb enough to forge a known spammer domain: still correct.)

I think perhaps I have this viewpoint because my focus is on my biggest
(ongoing) problem: what to do about the 99% of incoming mail that needs
to be rejected outright before it can get anywhere near a user.  Now, if
I get past that and focus on the remaining 1% that I'd actually like to
allow in, *then* I can see some reason to think about possible problems
with that: authentication, integrity, privacy, etc.

---Rsk

---

From: Franck Martin <[email-masked]>
Date: Wed, 21 Jan 2009 14:47:13 +1200 (FJT)

Hopefully domainkey will retire this year when yahoo moves from dk to dkim.

Anyone heard of dates?

----- Original Message -----
From: "John Levine" <[email-masked]>
To: [email-masked]
Sent: Wednesday, 21 January, 2009 1:04:27 PM (GMT+1200) Auto-Detected
Subject: Re: [Asrg] SPF, was where the message

R's,
John

---

From: Dave CROCKER <[email-masked]>
Date: Tue, 20 Jan 2009 19:32:41 -0800

> So dead that of hotmail.com, gmail.com, mac.com, apple.com, 
> microsoft.com, facebook.com, ebay.com only er..., all of them publish 
> SPF records.

It's not interesting who publishes SPF records.

It's interesting who reads them.

d/

-- 

   Dave Crocker

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 21 Jan 2009 10:30:20 +0000

>> This is a plea from Dave Crocker (author of rfc822) to take a
>> Thunderbird  extension that checks spf and dk, and add support for dkim.
>

Hmm, I'd have assumed that it checked the Received: headers to find an IP 
address that got an SPF pass with the content of the Return-Path: header. 
Of course, a spammer could always forge a suitable Received: header. I 
can't see that this could have worked 100% reliably. There would have been 
cases where are reliable warning could have been given, but they'd require 
a -all entry in the SPF record.

> The DK stuff didn't work at all AFAIK.

Well, that sucks. Is that because the DK signature was broken in transit?

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 21 Jan 2009 10:48:08 +0000

> On Tue, Jan 20, 2009 at 01:40:59PM +0000, Ian Eiloart wrote:

I guess that depends on the nature of the RBL. Some of them really are 
reputation systems. IP addresses get listed because someone has seen spam 
coming from them. Spamhaus' SBL is an example. If you don't agree that 
that's a reputation service, please explain.

Others aren't, Spamhaus' Policy Blocklist, for example is a list that IP 
address owners have added. I'm not sure why they don't just firewall the 
hosts, though. Perhaps so that some can be whitelisted. Of course each site 
can have its own local reputation policy that would trump a centralised 
reputation server - that's what whitelisting is about.

However, currently it's hard to know what to whitelist. There's only one 
widespread, easy to use mechanism for managing information about which IP 
addresses an organisation is likely to send messages from - that's SPF. OK, 
so if you wanted to be sure to get mail from me, you could whitelist my /24 
address block, but are you sure that I'd keep you updated if we outsourced 
our email?

And, you can't reliably whitelist my domain, because spammers could then 
easily bypass your filters. If I had a domain whitelist for my site, then 
spammers would be on a pretty good bet that forging any .ac.uk address 
would make their email more deliverable. It would also make it more likely 
to be read. So, I can't whitelist those domains unless the email is coming 
the right IP addresses. Only SPF can tell me that.

> dumb enough to forge a known spammer domain: still correct.)

Yes, the problem of course is when a spammer forges a domain that I'd like 
to trust. If I'm filtering mail from the domain of my chief funders, then 
false positives can be really painful. If I whitelist them, then spammers 
can easily bypass my filters. So what I'm discussing IS all about forgery.

> to be rejected outright before it can get anywhere near a user.

You mean you want to know how to identify it? Or what to do with it after 
you've identified it?

> Now, if

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 21 Jan 2009 11:08:05 +0000

> illegal in Australia, I think I've found a major recession-resistant

Phishing illegal? I don't know about Australia, but I don't think there are 
specific anti-phishing laws in the UK. Certainly, fraud perpetrated with 
information gained by phishing is illegal. It's fraud. It could be argued 
that the Phishing attempt itself is a form of fraud, but probably only if 
you actually do something with the information gained.

My point about trademarks is that registering a domain that could be easily 
confused with a trademark is illegal. Therefore, it's reasonable to code 
phishing defences that rely in whole or in part on detecting sender address 
domains that are similar, but not identical, to trademarked domains.

This means that technical measures to protect a domain from precise forgery 
can be supplemented with technical measures to protect near matches to 
those domains. Thus mitigating (not eliminating) the problem.

[Whether it's legal or not, it is a business opportunity. It's probably not 
totally recession-resistant because recession probably reduces the gains to 
be had.]

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 21 Jan 2009 11:16:39 +0000

> Same, you MUST implement domainkeys because yahoo likes it.
>

Actually, Yahoo have just implemented a new complaint feedback loop in the 
past few days. They ask for details of DomainKeys or DKIM, but don't seem 
to require either. Oh, actually, you can sign up without either DomainKeys 
or DKIM, but you only get feedback when they see a DK or DKIM signature.

<http://feedbackloop.yahoo.net:80/index.php>

Interestingly, you don't get feedback when Yahoo has put your mail into a 
spam mailbox. Only when it's marked as spam by the end user.

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 21 Jan 2009 11:28:39 +0000

> It's interesting who reads them.

That depends on whether you're deciding whether to publish them, or whether 
to read them.

If you're deciding whether to read them, then it's very interesting to know 
who's publishing them. Who else is reading them is of secondary importance.

If you're deciding whether to publish them, then it's interesting (but 
harder) to know who's reading them.

Of course, there's a cycle here. The value of SPF to readers increases as 
their trusted correspondents publish records, and to publishers as their 
trusted correspondents read records. The value to the world probably varies 
according to the product of the two probabilities: A that a received 
non-abusive email was received from an IP address with a positive SPF 
match, and B that the recipient checks the record. The value for me, 
depends only on those emails that pass my server. For inbound email, B is 
zero or one according to my choice. A is outside my control, but appears to 
be better than 30% already (for messages that pass IP address reputation 
checks).

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 21 Jan 2009 11:43:39 +0000

> John Levine wrote:
> security technology experts, it means too many things to know what is
> meant in a particular case.  To the rest of us, it carries too little
> precise meaning.
>
> technologies start with the authenticated identifier and then talk about
> what you can derive from it or how it might be spoofed or otherwise
> abused.  Since that is exactly what must be done with suspect messages,
> the model seems to make sense to (re-)apply for authentication.  I now
> believe it is exactly the wrong model.  That's a model that looks for the
> problem, the attack, the basis for suspicion.
>
>
>
> A very different model starts as a model of trust and specifies a table
> of who is part of the model.  In other words, it works in the oppsoite
> direction. When a message shows up, the only question is whether that
> message falls under that umbrella of trust.  If it doesn't, then we are
> done with that model, for that message.  Any further consideration of
> that message has nothing at all to do with the trust model.  The
> authenticated identifier is the sole, simple lookup string into that the
> model.  Either the identifier string works or it doesn't.

Agreed. That's why I'm discussing SPF or DKIM with a reputation service. In 
the first instance, my reputation service is going to be a local 
whitelisting mechanism. I'll probably have some domains whitelisted for my 
entire site. Perhaps all .ac.uk domains, for example. Then I'll allow users 
to whitelist domains and addresses that they trust. However, the 
whitelisting mechanism will rely on an SPF or DKIM pass.

So, as you say, I'll check that the sender address falls under my umbrella 
of trust - that it's in my whitelist. If not, I'll fall back to my old 
mechanisms. But, if it does fall under my umbrella of trust, I'm still 
going to check for either an SPF or a DKIM positive match. If I don't get 
that, then I fall back to my old mechanisms.

I hope that an trustworthy external reputation service will emerge that 
allows me to place others under my umbrella of trust, even when they're not 
listed locally.

I'll need some rules about who people can whitelist (and blacklist, too). 
For example, I don't think I'd allow anyone to whitelist hotmail.com as a 
domain, but I'd allow then to whitelist specific addresses. On the other 
hand, I'd allow them to whitelist the domain of a local school or local 
government organisation.

> Concerns about spoofing the identifier, re-purposing its use, and
> corrupting the content -- authentication, authorization and data and
> signature integrity -- are all valid for evaluating the underlying
> technology but they really are secondary to any serious discussion of the
> model.  After the briefest review to ensure that a particular technology
> has made specific and acceptable choices for the "security" questions,
> the questions should not burden debate about use of the technology.
> Instead we seem to find these questions hashed, rehashed and hashed
> again, as the focus of community debate.
>
> When the anti-abuse world discusses use of these authentication
> technologies, we need to stop being the anti-abuse world and start being
> the trust world.  The anti-abuse world really has the receiving assessor
> as an isolated island of suspicious.  In today's Internet, that's
> required.  But the trust world is quite different.  It is collaborative
> -- between signer and receiving assessor -- and deterministic, rather
> than vague and heuristic, and strictly the effort of the receiving
> assessor.
>
> Totally different model.  So we need a totally different approach to
> talking about it.
>
> d/

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 21 Jan 2009 17:05:12 +0100

>> [SPF] only works for the subset of mail that is sent from a fixed point

It works well also with a number of mailing lists. Alias expansions 
admittedly leaves something to be desired. However, considering that 
even rfc5321 screws up the term "forwarding" in that section, it 
should be tolerated in an experimental rfc...

> the processing rules could help a lot.

Would you please expand on that? Relaxing rules implies the knowledge 
that a message is being forwarded. Are you talking about whitelisting 
well known forwarders, or what?

> And I see promise in the use of the pending
(
Hm... the header's name suggests it is reporting already acquired 
results, as had been noted. I'm surprised Doug didn't propose an 
additional test more in tune with that spirit, e.g.

    Authentication-Results: example.com;
      dnsbl=pass zone=zen.spamhaus.org address=192.0.2.3
)
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 21 Jan 2009 08:12:09 -0800

> That depends on whether you're deciding whether to publish them, or 
> whether to read them.

For the purpose of assessing Internet utility, consumption is a better indicator 
than publication.

I understood this thread to be discussing utility.

Publication without consumption has zero utility.  So does consumption without 
publication, of course, but if there is a community wishing to consume, there 
will almost certainly come to be a community doing the publication.

In contrast we've had ample demonstration that that publication does not have 
any automatic implication about consumption.

d/
-- 

   Dave Crocker

---

From: John Leslie <[email-masked]>
Date: Wed, 21 Jan 2009 12:27:56 -0500

> John Leslie wrote:
>> processing by the receiving MTA is dubious. It's not the coding of
>> SPF records that breaks forwarding: it's the processing of them.
>> Relaxing the processing rules could help a lot.
> 

   I was intentionally vague...

   However, there are a limited number of ways that forwarding might be
shown in the trace headers, so it should be practical to determine that
a forwarding is documented (though possibly forged).

   We then have a quite different situation from what raw SPF processing
would indicate. Thus I claim the rules deserve to be relaxed (without
going into detail how).

   Forging headers to indicate forwarding which didn't happen indicates
evil intent, and should be practical to block-list like other spamming
IPs. Well-known forwarders could be whitelisted, enabling us to trust
their pre-forwarding headers. Et cetera...

>> And I see promise in the use of the pending Authentication-Results
>> header (though I must agree with Doug Otis that it would be stronger
>> if it included the IP address).
> 

   I'll let Doug speak for himself. I didn't propose such a thing
because I believe arguing over extensions would detract from getting
the basic header adopted.

   (I do believe that adding a resinfo listing the IP address is a
practical way to deal with SPF's choice to omit it from their resinfo.)

---

From: "J.D. Falk" <[email-masked]>
Date: Wed, 21 Jan 2009 10:31:15 -0700

> Same, you MUST implement domainkeys because yahoo likes it.

I haven't seen any reliable evidence that they're rejecting unsigned
messages.  Could you point us towards the announcement?

> Anyone heard of dates?

Yahoo!'s complaint feedback loop (re-launched yesterday, with help from
Return Path) routes based on either DK or DKIM, so that's progress.  They
aren't sharing dates for when they'll be working with DKIM in the rest of
the mailstream.

---

From: "J.D. Falk" <[email-masked]>
Date: Wed, 21 Jan 2009 10:33:50 -0700

> spam mailbox. Only when it's marked as spam by the end user.

That's why it's called a Complaint Feedback Loop.  All of the other North
American ISPs who provide feedback work basically the same way.

I've heard that there are a few European ISPs who provide filtering-based
feedback to each other, but not to just anyone who asks.

---

From: Rich Kulawiec <[email-masked]>
Date: Thu, 22 Jan 2009 11:07:20 -0500

> Agreed. That's why I'm discussing SPF or DKIM with a reputation service. 
> In the first instance, my reputation service is going to be a local  
> whitelisting mechanism. I'll probably have some domains whitelisted for 
> my entire site. Perhaps all .ac.uk domains, for example. Then I'll allow 
> users to whitelist domains and addresses that they trust. However, the  

Even if you go that route -- and I'll skip getting into its instrinsic
merits and problems here -- I think you shouldn't allow users to whitelist
*anything*, ever, without manual review by qualified personnel, for
at least two reasons I can think of.

First, users have very poor skills in this area.  (Not their fault,
really, it's not their gig.)  We can tell how poor their skills by
a number of methods, but I think the most obvious is: if they were
any good at it, then phishing would be an inconsequential problem.

Another way to tell is to monitor outbound SMTP and HTTP requests
and note which ones have being directed to known-fraudulent domains.
I see this constantly, even in environments where users have been
told ad infinitum to never reply to a suspected spam/phish, never
to follow any links in them, etc.  (In some environments, I block them.
I'm beginning to think that's a best practice, even in cases where
subsequently the real targets have acquired the phisher domains,
as I think they know never to use them.)

Second, how do you know that it's actually the users doing this?

I'm not saying that you shouldn't maintain something along the lines
of a whitelist, or an exempted-from-blacklist, or some other function.
I'm saying that nothing should ever go into that list until somebody who
knows about DNS and WHOIS and is appropriately paranoid has looked at it.

---Rsk

---

From: Rich Kulawiec <[email-masked]>
Date: Thu, 22 Jan 2009 23:47:53 -0500

> that's a reputation service, please explain.

I don't agree that's a reputation service.  It's a binary flag (on a per-IP
basis) saying "this address has been observed (at one or more observation
points) as sending spam".  And that's all it says -- and it only says it
about a subset of IP addresses (those seen emitting spam) and only about
a subset of those (those seen by Spamhaus-affiliated sensors) and only
about a subset of those (add some notion of "recently" -- an address
that spewed in the past may not be listed now).

That's highly useful information if my goal is to block spam, but it's
only marginally useful if my goal is to do anything else.

(BTW: there's only one RBL, and Spamhaus doesn't run it.  They run DNSBLs.)

Maybe I'm just quibbling over the definition of "reputation service",
or maybe my definition isn't broad enough.  But I don't think of DNSBLs
or RHSBLs that way, yet.  I'll mull it over.

> addresses an organisation is likely to send messages from - that's SPF. 
> OK, so if you wanted to be sure to get mail from me, you could whitelist 
> my /24 address block, but are you sure that I'd keep you updated if we 
> outsourced our email?

But this is not one of my pressing concerns: oh, it's not totally
off my radar, but it's far down on the priority scale.  I'll try
to explain below.

> Yes, the problem of course is when a spammer forges a domain that I'd 
> like to trust. If I'm filtering mail from the domain of my chief funders, 
> then false positives can be really painful. If I whitelist them, then 
> spammers can easily bypass my filters. So what I'm discussing IS all 
> about forgery.

If you are efficiently blocking spam, then this may be somewhat
of a non-issue.  (It pretty much is for me.)  Let me illustrate
with an example: traffic was presented a little while ago on
port 25 from 123.140.212.144:

	I could have sanity-checked the HELO.

	I could have run it through SPF or similar, but didn't.

	I could have waited for the data phase and run the content
	through SpamAssassin and/or ClamAV, but didn't.

	I could have looked up rDNS to see if it existed, but didn't.
	(Or, having it looked it up and found it to exist, checked
	the domain against various RHSBLs.  And/or for MX sanity.)

	I could have checked the IP address it against various DNSBLs.

	Instead, the mail system noted that it's in Korean IP space,
	which for that mail server is a 100% source of spam and a 0%
	source of mail.  So it was immediately rejected.

So maybe it was a phish with forged sender address at Paypal --
and maybe I could have figured that out via one of the methods
that have been discussed here.  But my point is that I didn't
need to, because I knew it was spam before getting that far.

Repeat this for myriad variations -- use of various DNSBLs, use of
the Spamhaus DROP list, various country allocations, thousands and
thousands of spammer domains, dynamic/generic IP and name space,
and so on.  What I've found is that if I'm sufficiently aggressive
about blocking spam sources (including pre-emptive blocks)
that I don't need to worry so much about what's in the spam -- forged
headers, bogus URLs, etc. -- because it never makes it to the point
where I have concern myself with any of that.

So my response to someone who says "I'm getting a lot of forged
traffic claiming to be from Paypal and I need an anti-forgery
method to figure that out" is "No, you need to be a lot more
aggressive about blocking spam.  AFTER you do that, you should
re-assess, see if this is still actually a problem for you, and
then, maybe, you might consider anti-forgery technology of one
sort or another, or even ad hoc local checks for some specific
cases, like maybe the credit union that serves your university."

Does this kinda clarify where I'm coming from?

> You mean you want to know how to identify it? Or what to do with it after 

The former -- because my approach to the latter is "issue a reject,
hang up, move on" in all cases.  (Although I should mention in passing
that DNS lookup failures get a 4XX 'cause maybe their DNS is hosed,
maybe mine is hosed, maybe transport is fubar.)

The former is tougher, because -- despite my aggressive approach
to spam, I don't want to deal with a high FP rate.  I've come to
conclusion that one approach which seems to work boils down to
"know your email": study traffic patterns, inbound and outbound.
Every mail server (that I've ever looked at) has different characteristics,
and if you can figure out what they are, then you can twiddle the
knobs in very server-specific ways that minimize FN and FP at the
same time.  (See example above, which clearly would not work at
all on some other servers -- say for a research university.)

Of course, this takes time and patience -- but I'll argue that
we should be trying to extract the most from the methods we already have
(like the ones I tossed out above), that we understand fairly well,
and that we know work on a large scale in production environments,
before we try to invent and deploy new methods.

---Rsk

---

From: Ian Eiloart <[email-masked]>
Date: Fri, 23 Jan 2009 10:40:58 +0000

> On Wed, Jan 21, 2009 at 10:48:08AM +0000, Ian Eiloart wrote:
>> reputation systems. IP addresses get listed because someone has seen
>> spam  coming from them. Spamhaus' SBL is an example. If you don't agree
>> that   that's a reputation service, please explain.
>
> I don't agree that's a reputation service.  It's a binary flag (on a
> per-IP basis) saying "this address has been observed (at one or more
> observation points) as sending spam".  And that's all it says -- and it
> only says it about a subset of IP addresses (those seen emitting spam)
> and only about a subset of those (those seen by Spamhaus-affiliated
> sensors) and only about a subset of those (add some notion of "recently"
> -- an address that spewed in the past may not be listed now).
>

I stand corrected.

> or RHSBLs that way, yet.  I'll mull it over.

OK, I see where you're coming from now. I guess we might agree that DNSBLs 
collectively provide a minimalist reputation service.

I do think that DNSBLs speak to the reputation of some email emitters. In 
that respect, I'd argue that even a single data point constitutes a comment 
on reputation, and it it's available to me then it's providing me with a 
service - even if the service is no good. However, even a single datapoint 
could be a useful service if it stopped lots of spam for me.

Collectively, DNSBLs provide more than binary information about a single IP 
address. For example, you could consult a dozen of them, and compare a 
weighted sum of their responses against a threshold determined in your 
local anti-spam policy.

I disagree that they say nothing about addresses that they don't list. 
Silence is a comment, in this context. Provided that list policies apply to 
all IP addresses equally, then they are commenting on all IP addresses. 
However, if a DNSBL had a policy (explicit or implicit) that it would never 
list a certain address range, then the service ceases to be comprehensive. 
It doesn't cease to be a service, though.

>>  widespread, easy to use mechanism for managing information about which
>> IP  addresses an organisation is likely to send messages from - that's
>> SPF.  OK, so if you wanted to be sure to get mail from me, you could
>> whitelist  my /24 address block, but are you sure that I'd keep you
>> updated if we  outsourced our email?
>
>> like to trust. If I'm filtering mail from the domain of my chief
>> funders,  then false positives can be really painful. If I whitelist
>> them, then  spammers can easily bypass my filters. So what I'm
>> discussing IS all  about forgery.
>

Yes, it does. However, I'm an admin for a University with students from 
most countries in the world, and academics that work in most countries in 
the world, and studying every topic under the sun - including SPAM! So, 
it's very difficult for me to be that aggressive. I certainly block IP 
addresses according to country allocation.

That's why I need more information about who the IP addresses belong to. 
Without that information, and with the prevalance of sender address 
forgery, the IP address is the only real information that I have about a 
message before it's too late to reliably apply recipient specific filtering.

>> You mean you want to know how to identify it? Or what to do with it
>> after  you've identified it?
>
> Every mail server (that I've ever looked at) has different
> characteristics, and if you can figure out what they are, then you can
> twiddle the knobs in very server-specific ways that minimize FN and FP at
> the same time.  (See example above, which clearly would not work at

---

From: Ian Eiloart <[email-masked]>
Date: Fri, 23 Jan 2009 10:43:30 +0000

> On 21/01/2009 04:16, "Ian Eiloart" <[email-masked]> wrote:
>

Yes, I guess it makes sense, but it's not something that I'd really thought 
about.

---

From: Ian Eiloart <[email-masked]>
Date: Fri, 23 Jan 2009 10:48:05 +0000

> Alessandro Vesely <[email-masked]> wrote:
>> John Leslie wrote:
>>

Blech. Why not just let them rewrite the sender address. People just should 
not be encouraged to send email with return-paths in domains that don't 
belong to them. It simply postpones the day when we can hold senders 
accountable for their traffic.

---

From: Ian Eiloart <[email-masked]>
Date: Fri, 23 Jan 2009 11:00:26 +0000

> On Wed, Jan 21, 2009 at 11:43:39AM +0000, Ian Eiloart wrote:

Well, I don't think we have the staff capacity to do that. I guess we might 
require review when entire domains are whitelisted, and I guess that if 
we're permitting one person to whitelist a domain, then we should permit 
anyone to whitelist that domain.

Of course, if a domain doesn't have an SPF or DKIM record, then we won't 
let anyone whitelist it or any address in that domain. And, if a message 
doesn't have a positive SPF or DKIM match, we'll ignore the whitelist entry 
- or perhaps warn the recipient if we don't accept the message.

I don't know how I could be a better judge than a user about whether they 
want to whitelist a specific email address. However, we could present them 
with a warning if the try to list an address (in a domain)? with a poor 
rating in our reputation service. The point here is that permitting address 
A to email address B only exposes one person to risk - the person doing the 
whitelisting.

Oh, and I'd never allow any email (other than to postmaster@ or abuse@) to 
bypass our malware filter. That would turn a spam threat to an individual 
into a threat to our network.

---

From: Ian Eiloart <[email-masked]>
Date: Fri, 23 Jan 2009 13:04:53 +0000

> For the purpose of assessing Internet utility, consumption is a better
> indicator than publication.

Like I said, you need both. But, I'm not interested in "Internet utility", 
whatever that might be. I'm interested in utility to me. I presume that 
what's useful to me might also be useful to some others.

> I understood this thread to be discussing utility.

There are many kinds.

> Publication without consumption has zero utility.  So does consumption
> without publication, of course, but if there is a community wishing to
> consume, there will almost certainly come to be a community doing the
> publication.

Except that there's no obvious way to advertise the fact that you're 
consuming. We know that there's widespread publication. We don't know 
whether there's widespread consumption - unless someone has stats gathered 
from DNS servers. Even if we had stats from DNS servers, it's hard to know 
what anyone is doing with the information. Widespread selective 
whitelisting would be hard to detect, for example.

>
> In contrast we've had ample demonstration that that publication does not
> have any automatic implication about consumption.
>
> d/

---

From: Rich Kulawiec <[email-masked]>
Date: Fri, 23 Jan 2009 08:08:41 -0500

> Well, I don't think we have the staff capacity to do that.

It's not really that difficult, if well-organized.  Having done this
in practice for a number of years, I can report that my experience has
been that nearly all such requests can be denied on inspection -- to
the point where I've scripted much of it.  For example: all requests
to whitelist the incompetently-managed and spammer-infested domain
"yahoo.com" are automatically denied.  All requests to whitelist domains
that have been placed in the local blacklist (which is done only after
considerable study) are denied, since there is never any reason to
delist any known spammer.  The ones that actually merit attention and
might result in action are almost always transient, accidental cases:
e.g., example.com is a long-time source of non-spam mail but fubar'd
their DNS while making a change, and we need to exempt them from
DNS checks while they work it out.

One thing that's quite revealing is how many users ask for obvious
phish domains to be whitelisted.  Were the process automated, without
human review, any number of fake eBay and Chase and Visa &etc. domains
would have long since been repeatedly whitelisted.

I'm skeptical about the merits of doing per-address whitelisting, even
though I do some of it.  On the one hand, it accomodates people who are
stuck with poorly-run systems and networks.  On the other hand, it removes
much of their motivation to agitate for a change in that situation.
A compromise that I've used, some of the time, is to provide them with
service, but slower/degraded service, and explain to them that I'd be
delighted to extend to them the same privileges as others enjoy, but
that their operation needs to step up the responsibility that goes
along with that.  Sometimes this works, other times it fails.

---Rsk

---

From: Ian Eiloart <[email-masked]>
Date: Fri, 23 Jan 2009 13:16:49 +0000

> On Fri, Jan 23, 2009 at 11:00:26AM +0000, Ian Eiloart wrote:

Yes, yes, yes. But I want to do something smarter that that, with more 
granularity. OK, I'm not going whitelist "yahoo.co.uk" or any of the other 
well known ESPs. However, I'm not going to spend staff time (there's me 
versus 16,000 users here) on determining whether [email-masked] is really 
a friend of [email-masked]

> much of their motivation to agitate for a change in that situation.

I've a lot of sympathy with that. And, I currently don't do any 
whitelisting, and that's part of my reasoning. However, it does mean that 
spammers are making it harder to create working email communications 
systems, and that our anti-spam systems are often merely RFC compliance 
enforcement systems. I'd rather be fighting evil than incompetence!

---

From: John Leslie <[email-masked]>
Date: Fri, 23 Jan 2009 09:13:19 -0500

> --On 21 January 2009 12:27:56 -0500 John Leslie <[email-masked]> wrote:
> 

   The point I was attempting to make is that SPF records _can_ accurately
reflect sender policy, while SPF processing will incorrectly indicate a
violation of it.

   As things stand in SPF, folks end up publishing less-correct records
in an attempt to tune to a more satisfactory result.

> Blech. Why not just let them rewrite the sender address.

   You, of course, are welcome to do whatever you want with SPF records;
I happen to dislike punishing MTAs for following the SMTP specs.

   But please understand that strict SPF processing hasn't yet stopped
forwarding MTAs from documenting the forwarding according to spec
rather than rewriting addresses the way you want them to. Do you really
believe this will change?

> People just should not be encouraged to send email with return-paths
> in domains that don't belong to them. It simply postpones the day when
> we can hold senders accountable for their traffic.

   Unfortunately, that is what the SMTP RFCs call for: if you don't
like it, you should be seeking consensus to change them.

   Furthermore, you seem to be confusing "people who send email" with
MTAs which process it. The return-path is intended to be the "best"
address for notifications. As things currently stand, MTAs are in no
position to second-guess whether some other address would be "better".

---

From: Ian Eiloart <[email-masked]>
Date: Fri, 23 Jan 2009 14:22:49 +0000

> Ian Eiloart <[email-masked]> wrote:

Something has to. Every false positive I encounter says so. My guess is 
that deployment of spf will grow gradually to the point where people start 
taking notice, and one day it will be normal, and people will start to 
notice that they're having difficulty forwarding email without, for 
example, using SRS.

> like it, you should be seeking consensus to change them.

I'm sort of doing that here. DKIM might be a suitable solution for this, 
but it's something you can only process after seeing the data.

> position to second-guess whether some other address would be "better".

No, I think that's what SRS was for, though.

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 23 Jan 2009 20:02:10 +0100

> reflect sender policy, while SPF processing will incorrectly indicate a

That's quite correct. I say "quite" because SPF provides for various 
levels of results, without mandating any particular behavior. However, 
the quickest behavior, reject on fail, does not allow to examine the 
message headers.

> in an attempt to tune to a more satisfactory result.

Yup. With rules like "?ip4:80.0.0.0/5" one can succinctly "neutralize" 
many IPv4 addresses, possibly excluding forwarders located elsewhere. 
For geographically characterized sites, half of the world would still 
be much less than all of it.

> [...]

I hope it will. Headers that document forwarding are unmanageable. 
Some include full explanations in the body, but most don't. Wild 
forwarding breaks more than SPF: it breaks most countries' privacy 
laws by not letting a recipient know where an alias was expanded from. 
More mumblings at http://fixforwarding.org/wiki/privacy_laws
_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 26 Jan 2009 10:27:17 +0000

--On 23 January 2009 20:02:10 +0100 Alessandro Vesely <[email-masked]> 
wrote:

>>    The point I was attempting to make is that SPF records _can_
>>    accurately reflect sender policy, while SPF processing will
>> incorrectly indicate a violation of it.
>

Beware of the word "quite". It can mean "totally" or "partly", depending on 
context. For example, the term "quite correct" means "absolutely correct", 
but the term "quite good" could mean "good, but not very good"! The OED 
thinks that the "partly" meaning is rare in North America, though.

---

From: Bill Cole <[email-masked]>
Date: Mon, 26 Jan 2009 22:54:43 -0500

John Levine wrote, On 1/20/09 9:53 AM:
[...]
> SPF's fans, that last part is a fundamental design failure of SPF, not

I should preface this by saying that I deeply regret my public and private
discussions of the embryonic source concepts that spawned SPF and Sender-ID
and if in any way I helped evangelize a single SPF fan in my 2000-2004
delusional state, I am terribly sorry and beg forgiveness. I was so wrong.
My only excuse is that a lot of other smarter and/or more influential people
were also wrong...

The reason that SPF is here to stay is that it is good enough authentication 
for most of the mail that most receivers and senders care the most about. A 
lot of people get significant quantities of mail they care about via 
forwarding systems or have to send through paths that can't get them a SPF 
'+' anywhere that matters without +all, but we are a minority. Legitimate 
bulk senders, consumer retail ISP's, and non-bulk business mail senders can 
all make their outbound mail pass SPF at their border, and the largest (i.e. 
consumer ISP) and pickiest (i.e. corporate) receiving systems can get some 
benefit by using SPF in some way on some of their incoming mail. The people 
who encounter significant breakage in SPF are an elite class, including a 
large fraction of the people who actually work with mail technology directly 
(i.e. *US*) but having created this breakage with our eyes open and having 
given the masses a tool that mostly works for them, we are stuck with SPF 
until something that works significantly better for them with similar or 
less effort can replace it.

> you tell short of manually looking up WHOIS registrations?

Which is half of why DKIM is not a SPF-killer. It shares a key gap with SPF 
that actually makes a difference to the masses: some way to tell whether an 
authenticated sender is actually a good sender. The other half is that it is 
  significantly harder to deploy on both ends. The fact that it is more 
robust than SPF when dealing with mail that takes interesting paths can't 
make up for the big gap: it is only an authentication tool, and the 
authentication only goes as far as demonstrating the cooperation of a mail 
admin and a dns admin (or someone who can crack the relevant mail and DNS 
systems) with a sender.

It seems to me that if anyone ever devises a trustworthy and broadly 
deployable/applicable shared *authorization* layer for mail, SPF and DKIM 
will both be too weak on other grounds to be the dominant authentication 
mechanism. If there were such a thing as a trustworthy domain vouching 
system to tell me that I can be sure that mail definitely from a paypal.com 
sender is not phishing and mail definitely from a pay-pal.com sender is, 
then I don't see why I'd use DKIM to do the authentication instead of 
nagging the paypal.com folks to sign their mail with S/MIME so that I don't 
have to support 2 signing technologies.

---

From: Bill Cole <[email-masked]>
Date: Mon, 26 Jan 2009 23:16:55 -0500

> reflect sender policy, while SPF processing will incorrectly indicate a

There's also a subtle variation on that...

It was discussed (even here) in the early development of SPF that a trailing 
'-all' can and perhaps should  be read as a 'do not forward' policy 
statement by a domain owner. That was not considered a valuable feature by 
the people who ended up promoting SPF, but there are people who publish such 
records with the intent of expressing such a policy.

Whether such a policy should be within the authority of a domain owner is a 
knotty question.