Subject: [Asrg] What are the IPs that sends mail for a domain?

From: Franck Martin <[email-masked]>
Date: Wed, 17 Jun 2009 10:20:44 +1200 (FJT)

I recently encountered the following question/problems. 

I have a mail server and one of my users complains he is not receiving emails from a domain. How do I find if I have blocked the domain from sending to my server. Meaning, knowing the domain name of the sender, how do I find the IPs from where the mail could be sent from. It seems that SPF is the only tool to provide that answer? 

In another related problem, which is linked to IPv6 and RBL. Buidling an IPv6 RBL could lead to a huge database. Sure you can alleviate by using "wildcards", but why not use the reverse DNS resolution to add a TXT record associated to the IP to indicate the IP is the one of a mail server? So any IP that does not have this record would be blocked for SMTP. As IPv6 is not used for SMTP (or barely), this could be made mandatory for IPv6 and optional for IPv4. An MUA could talk to an MTA on port 25 because we know the the etwork range of the MUA or the alternative is to use the new mail submit port.

---

From: John Levine <[email-masked]>
Date: 16 Jun 2009 22:55:43 -0000

>How do I find if I have blocked the domain from sending to my server. Meaning, knowing the
>domain name of the sender, how do I find the IPs from where the mail could be sent from. It
>seems that SPF is the only tool to provide that answer? 

Unless you have previous mail from the domain, I would agree SPF is your best bet.

>In another related problem, which is linked to IPv6 and RBL. Buidling an IPv6 RBL could lead
>to a huge database. Sure you can alleviate by using "wildcards", but why not use the reverse
>DNS resolution to add a TXT record associated to the IP to indicate the IP is the one of a
>mail server? So any IP that does not have this record would be blocked for SMTP.

We've had a variety of proposals to identify mail client hosts.  See http://mipassoc.org/csv/

R's,
John

---

From: SM <[email-masked]>
Date: Tue, 16 Jun 2009 16:01:25 -0700

At 15:20 16-06-2009, Franck Martin wrote:
>I have a mail server and one of my users complains he is not 
>receiving emails from a domain. How do I find if I have blocked the 
>domain from sending to my server. Meaning, knowing the domain name 
>of the sender, how do I find the IPs from where the mail could be 
>sent from. It seems that SPF is the only tool to provide that answer?

You can use the mail log to track down whether you blocked the 
domain.  The information might not be available if you are blocking 
by IP address.  You could also contact the postmaster at the other end.

Regards,
-sm

---

From: "John Johnson" <[email-masked]>
Date: Tue, 16 Jun 2009 18:03:34 -0500

> Unless you have previous mail from the domain, I would agree SPF is your best bet.
   I would also add that if your e-mail is important, having good
logging on your server
   of when the domain or ip was blocked can help speed up rectifying the
problem.

   Yes, it's after the block occurred - but so was the complaint.

---

From: Daniel Feenberg <[email-masked]>
Date: Tue, 16 Jun 2009 19:17:41 -0400 (EDT)

On Wed, 17 Jun 2009, Franck Martin wrote:

> I have a mail server and one of my users complains he is not receiving 
> emails from a domain. How do I find if I have blocked the domain from 
> sending to my server. Meaning, knowing the domain name of the sender, 
> how do I find the IPs from where the mail could be sent from. It seems 
> that SPF is the only tool to provide that answer?
>
> In another related problem, which is linked to IPv6 and RBL. Buidling an 
> IPv6 RBL could lead to a huge database. Sure you can alleviate by using 
> "wildcards", but why not use the reverse DNS resolution to add a TXT 
> record associated to the IP to indicate the IP is the one of a mail 
> server? So any IP that does not have this record would be blocked for 
> SMTP. As IPv6 is not used for SMTP (or barely), this could be made 
> mandatory for IPv6 and optional for IPv4. An MUA could talk to an MTA on 
> port 25 because we know the the etwork range of the MUA or the 
> alternative is to use the new mail submit port.

I predict that no significant amount of mail ever originates from IPV6. 
Because it would be impossible to maintain a DNSBL for IPV6, I expect that 
enough sites will decline all IPV6 mail that it won't pay to send from it.
Consider that because a spammer could (spoof) a different IPV6 address for 
every message, even a different 48 bit block for every messages, MTAs will 
be left with only content analysis for spam blocking. I don't expect IPV4s 
will ever be so scarce that enough MTAs will start using them out of 
necessity - ISPs will give each customer 4 IPV4 addresses with their 
million address IPV6 range, and customers will use those 4 addresses for 
the things that really need IPV4 - such as internet facing MTAs.

Consider also the difficulty facing the first IPV6 only MTA. It's 
connectivity will be very low, even compared to the worst possible 
allocation of a former spammer block in IPV4. It is one thing to put up a 
little used IPV6 web site - there isn't much of a penalty for adding IPV6 
to IPV4. And eventually some special purpose websites will be IPV6 only, 
those that know their clients are IPV6 only. But there isn't any point in 
a public-facing IPV6 MTA without an IPV4 alternative. And since the IPV4 
alternative can serve the IPV6 clients just as well, it will never be very 
usefull to add IPV6 support.

This is a prediction, not a prescription.

Daniel Feenberg

---

From: Douglas Otis <[email-masked]>
Date: Tue, 16 Jun 2009 17:24:03 -0700

On Jun 16, 2009, at 3:55 PM, John Levine wrote:

>> How do I find if I have blocked the domain from sending to my  
>> server. Meaning, knowing the domain name of the sender, how do I  
>> find the IPs from where the mail could be sent from. It seems that  
>> SPF is the only tool to provide that answer?
>

This is not your only bet.  Many SPF records include the term MX and,  
when not found, even default to using MX/24.

>> In another related problem, which is linked to IPv6 and RBL.  
>> Buidling an IPv6 RBL could lead to a huge database. Sure you can  
>> alleviate by using "wildcards", but why not use the reverse DNS  
>> resolution to add a TXT record associated to the IP to indicate the  
>> IP is the one of a mail server? So any IP that does not have this  
>> record would be blocked for SMTP.
>

The CSV effort proved most providers do not want their MTAs identified  
as belonging to them, even when it could improve email acceptance.   
This might be especially true now after their support staff has been  
reduced.

Reverse DNS is already causing a large amount of resources to be  
wasted by the shabby state of the reverse name space.  Incorrectly  
configured RFC 2317 delegation, and many non-functional servers are  
causing MTAs to rapidly become resource limited when making reverse  
checks.   In addition, when your customers conduct business with Asia,  
they may not be happy to find email is being lost as a result of  
geographic differences of opinion about the role that reverse DNS  
might play with email.

IMHO, all outbound MTAs should be required to return CVS records for  
their EHLO name and offer MX records for their inbound.  A mandate  
that required MX (inbound) or CVS (outbound) records would greatly  
help in identifying non-abusive email sources against a backdrop of  
hundreds of millions of bot-net controlled drones spewing email.   
Systems may soon use ACLs as a means to white-list safe MTAs.  Perhaps  
the world is a few years from having to go to that extreme.

-Doug

---

From: Lyndon Nerenberg <[email-masked]>
Date: Tue, 16 Jun 2009 19:55:45 -0600

On Tue, 2009-06-16 at 17:24 -0700, Douglas Otis wrote:
> their EHLO name and offer MX records for their inbound.

Doug, are you sure that's what you meant to say? The sentence is a bit
ambiguous. Are you really saying any host that sends mail (is an SMTP
client) MUST also host an listed SMTP server? 

--lyndon

---

From: Bill Cole <[email-masked]>
Date: Tue, 16 Jun 2009 23:27:27 -0400

> client) MUST also host an listed SMTP server?

I can't testify to what he meant, but I think what he is actually saying is 
that if you have a machine that says "EHLO some.name" then there should be 
both a MX record for some.name and a SRV record for _client._smtp.some.name 
(i.e. a CSV/CSA record).

That doesn't mean requiring inbound SMTP on every outbound, it means 
requiring an affirmation in DNS that a name can be used in EHLO by a 
particular IP address and a way to get mail to the responsible party for the 
machine(s) using that name in EHLO. This is an admirable goal. A weaker goal 
would be to get people running non-spamming mail servers to follow the 
existing accepted standard of using a valid resolvable FQDN in EHLO.

---

From: Franck Martin <[email-masked]>
Date: Wed, 17 Jun 2009 15:33:22 +1200 (FJT)

Knowing that mail servers are not deployed on IPv6, what would it take to make all these requirements mandatory for IPv6 and start with a better infrastructure than on IPv4?

----- Original Message -----
From: "Bill Cole" <[email-masked]>
To: "Anti-Spam Research Group - IRTF" <[email-masked]>
Sent: Tuesday, 16 June, 2009 8:27:27 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna
Subject: Re: [Asrg] What are the IPs that sends mail for a domain?

_______________________________________________
Asrg mailing list
[email-masked]
http://www.irtf.org/mailman/listinfo/asrg

---

From: Bill Cole <[email-masked]>
Date: Tue, 16 Jun 2009 23:36:37 -0400

Franck Martin wrote, On 6/16/09 6:20 PM:
> how do I find the IPs from where the mail could be sent from.

There is no reliable way to do so.

> It seems

Only partially. SPF cannot be considered reliable, since not all domains 
publish records and some publish inaccurate records.

There have been other proposed approaches that may have some deployment:

CSV/CSA: http://mipassoc.org/csv/draft-ietf-marid-csv-csa-02.html
DRIP: http://tools.ietf.org/html/draft-brand-drip-02

> alternative is to use the new mail submit port.

Similar proposals have been made before, and I'm pretty sure one such has 
been made on this list although I can't find proof of that at present.

There's always some degree of resistance to putting information into the 
reverse zone because it is frequently under different control than related 
forward zones and can be a chore to get set or changed. There are also 
concerns about loading up new sorts of records into the reverse zone because 
it is a simpler tree that has traditionally had light query volume, and the 
existing systems may not be prepared to handle an extra query down the 
reverse tree for every SMTP connection.

That said, I think that adding DNS records that map specific network 
addresses to their legitimate behaviors in a generalized model would be a 
positive advance.

---

From: der Mouse <[email-masked]>
Date: Tue, 16 Jun 2009 23:58:27 -0400 (EDT)

> Knowing that mail servers are not deployed on IPv6,

They're not?  Mine has been for years.  netbsd.org's MX host is
v6-reachable and I think it has been for years too.  freebsd.org's
ditto.  And two of icann.org's five MX hosts are v6-reachable too and
probably have been for quite a while.

Where did you get the idea mailservers aren't deployed on v6?

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		[email-masked]
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

---

From: Franck Martin <[email-masked]>
Date: Wed, 17 Jun 2009 16:27:58 +1200 (FJT)

well, I have still enough fingers to count them....

---

From: Bill Cole <[email-masked]>
Date: Wed, 17 Jun 2009 01:14:02 -0400

Franck Martin wrote, On 6/16/09 11:33 PM:
> Knowing that mail servers are not deployed on IPv6, what would it take to
> make all these requirements mandatory for IPv6 and start with a better
> infrastructure than on IPv4?

How do you make anything mandatory on the net?

RFC 821 is one of a handful of Internet Standards, and it is violated 
routinely by spammers and non-spammers for no better reason than that they 
never bothered to read it. That is possible because the major MTA's are 
functional when misconfigured (e.g. with a bogus name for EHLO/HELO use) and 
by default tolerate clients which violate standards.

The only way anything can be functionally mandatory for email transport is 
if major MTA's will not work unless configured to comply and by default will 
not interoperate with clients that do not comply. RFC's are great, but they 
do not enforce themselves. If the big freemail providers and sites running 
Sendmail, Exchange, and Postfix generally accept mail from non-compliant 
clients, there will be a lot of non-compliant clients. To make good behavior 
mandatory, bad behavior has to break with enough frequency that it's easier 
to comply than negotiate exemptions.

> ----- Original Message ----- From: "Bill
> Cole"<[email-masked]> To: "Anti-Spam Research Group -
> IRTF"<[email-masked]> Sent: Tuesday, 16 June, 2009 8:27:27 PM GMT +01:00
> Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: Re: [Asrg]
> What are the IPs that sends mail for a domain?
>
> I can't testify to what he meant, but I think what he is actually saying
> is that if you have a machine that says "EHLO some.name" then there
> should be both a MX record for some.name and a SRV record for
> _client._smtp.some.name (i.e. a CSV/CSA record).
>
> particular IP address and a way to get mail to the responsible party for
> the machine(s) using that name in EHLO. This is an admirable goal. A
> weaker goal would be to get people running non-spamming mail servers to
> follow the existing accepted standard of using a valid resolvable FQDN in
> EHLO.
>
>
> _______________________________________________ Asrg mailing list
> [email-masked] http://www.irtf.org/mailman/listinfo/asrg
> _______________________________________________ Asrg mailing list
> [email-masked] http://www.irtf.org/mailman/listinfo/asrg

---

From: Franck Martin <[email-masked]>
Date: Wed, 17 Jun 2009 17:27:52 +1200 (FJT)

Sure, it is the the be strict in what you send, lenient in what you receive.

If we don't specify some RFC/BCP to specify how SMTP over IPv6 should be negotiated, then no one will follow.

We could say something like all emails on IPv6 must have a DKIM signature, have RDNS helo, etc... as there is not much of an implementation with IPv6, there is a chance for these practices to be adopted from day one...

---

From: John Levine <[email-masked]>
Date: 17 Jun 2009 08:51:06 -0000

>negotiated, then no one will follow.

The IETF is amazingly resistant to making v6 SMTP different from v4 SMTP
in any way.

In particular, I suggested that they not have a rule for fallback to
AAAA in the absence of MX.  The rationale is straightforward: most
hosts with AAAA (and indeed A) records are not mail servers, people
need to add new DNS records anyway for v6 so the incremental effort to
install MX records is quite small, and it'll make mail more reliable
by making it easier to tell when a domain doesn't receive mail.  They
all said too late, there are already a handful of v6 mail hosts.
Sigh.

R's,
John

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 17 Jun 2009 11:29:01 +0100

--On 16 June 2009 17:24:03 -0700 Douglas Otis <[email-masked]> wrote:

>
> The CSV effort proved most providers do not want their MTAs identified as
> belonging to them, even when it could improve email acceptance.  This
> might be especially true now after their support staff has been reduced.

It'll probably depend on how much difference it makes to email acceptance. 
The harder it is to deliver email without some assurance that the sender 
isn't spoofed, the better.

> Reverse DNS is already causing a large amount of resources to be wasted
> by the shabby state of the reverse name space.  Incorrectly configured
> RFC 2317 delegation, and many non-functional servers are causing MTAs to
> rapidly become resource limited when making reverse checks.   In
> addition, when your customers conduct business with Asia, they may not be
> happy to find email is being lost as a result of geographic differences
> of opinion about the role that reverse DNS might play with email.
>
> their EHLO name and offer MX records for their inbound.  A mandate that
> required MX (inbound) or CVS (outbound) records would greatly help in
> identifying non-abusive email sources against a backdrop of hundreds of
> millions of bot-net controlled drones spewing email.  Systems may soon
> use ACLs as a means to white-list safe MTAs.  Perhaps the world is a few
> years from having to go to that extreme.

It's not a binary thing, though.

-- 
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 17 Jun 2009 11:45:30 +0100

> all said too late, there are already a handful of v6 mail hosts.

But, do they have MX records? If yes, there's not a problem. Do you receive 
any email from them? If no, there's not a problem.

If a good chunk of the world implemented the rule that you've suggested 
(perhaps "*.ac.uk" or  "*.gov" domains, or just gmail), then we'd be in a 
good place. It might not be too late for some leadership initiative to 
actually make a difference outside the IETF.

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 17 Jun 2009 11:40:06 +0100

--On 17 June 2009 01:14:02 -0400 Bill Cole <[email-masked]> 
wrote:

> routinely by spammers and non-spammers for no better reason than that
> they never bothered to read it.

Well, parts of it are. The rest is mandatory for the purely practical 
reason that you can't deliver email without obeying those parts. For 
example, to send email to someone, it IS mandatory to give their email 
address in a RCPT command.

How do you make other parts mandatory? Well, it's a long and arduous task, 
but the steps look like this:

1. make sure that the bulk of client MTA's behave correctly
2. start basing reputation scores on failure to respect the standard
    this can take several forms: refusal to whitelist non-compliant 
senders, incrementing spam scores, rejecting mail

As the deliverability of non-compliant email drops, the proportion of 
senders complying will increase. A virtuous circle takes us to a world 
where everybody is compliant. Eventually, even the spammers comply. So, 
it's just an arms race in some cases, but in other cases we may have 
regained some real value. For example, if respecting SPF were universal 
(with fixes for forwarding), then backscatter would not be a problem.

> That is possible because the major MTA's
> are functional when misconfigured (e.g. with a bogus name for EHLO/HELO
> use) and by default tolerate clients which violate standards.
>
> The only way anything can be functionally mandatory for email transport
> is if major MTA's will not work unless configured to comply and by
> default will not interoperate with clients that do not comply. RFC's are
> great, but they do not enforce themselves. If the big freemail providers
> and sites running Sendmail, Exchange, and Postfix generally accept mail
> from non-compliant clients, there will be a lot of non-compliant clients.
> To make good behavior mandatory, bad behavior has to break with enough
> frequency that it's easier to comply than negotiate exemptions.
>
>

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 17 Jun 2009 11:48:49 +0100

--On 16 June 2009 19:17:41 -0400 Daniel Feenberg <[email-masked]> wrote:

> Because it would be impossible to maintain a DNSBL for IPV6, I expect
> that enough sites will decline all IPV6 mail that it won't pay to send
> from it.
> Consider that because a spammer could (spoof) a different IPV6 address
> for every message, even a different 48 bit block for every messages, MTAs
> will be left with only content analysis for spam blocking.

Which is why reputation services need to be based on sender domains, not IP 
addresses. Users can then whitelist as required, and use of IP 
addresses/domains without good positive reputation won't work very well.

The advantage of IPV6, of course, is that you'll never have to share an IP 
address with someone with poor reputation.

> I don't expect
> IPV4s will ever be so scarce that enough MTAs will start using them out
> of necessity - ISPs will give each customer 4 IPV4 addresses with their

---

From: Dotzero <[email-masked]>
Date: Wed, 17 Jun 2009 08:25:28 -0400

On Tue, Jun 16, 2009 at 6:20 PM, Franck Martin<[email-masked]> wrote:
> sending to my server. Meaning, knowing the domain name of the sender, how do
> I find the IPs from where the mail could be sent from. It seems that SPF is
> the only tool to provide that answer?
>

One approach that might help you is to go to senderscore.org (from
ReturnPath). Register for a free account and then enter in the domain
name. For example, when I enter in avonsys.com it shows me that there
is one IP address sending mail for that domain - 76.203.192.33 with a
hostname of adsl-76-203-192-33.dsl.rcsntx.sbcglobal.net.

Hope this helps.

> "wildcards", but why not use the reverse DNS resolution to add a TXT record
> associated to the IP to indicate the IP is the one of a mail server? So any
> IP that does not have this record would be blocked for SMTP. As IPv6 is not
> used for SMTP (or barely), this could be made mandatory for IPv6 and
> optional for IPv4. An MUA could talk to an MTA on port 25 because we know
> the the etwork range of the MUA or the alternative is to use the new mail
> submit port.
>

---

From: Seth <[email-masked]>
Date: Wed, 17 Jun 2009 09:24:18 -0400 (EDT)

Daniel Feenberg <[email-masked]> wrote:

> I predict that no significant amount of mail ever originates from
> IPV6. Because it would be impossible to maintain a DNSBL for IPV6,

But a DNSWL would be just as easy as IPV4.  So all it would take is
half a dozen of the largest senders and receivers (google, microsoft,
outblaze, aol, etc.) to use IPV6 among themselves and your statement
would already be wrong.

Then, if they set up a method to allow others to tentatively whitelist
IPV6 addresses, they could switch to more draconian blocking/checking
of IPV4 space over time.

Seth

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 17 Jun 2009 15:33:08 +0200

> _client._smtp.some.name (i.e. a CSV/CSA record).

However, the standard requires that it says "EHLO host-at.some.name". 
It is a seemingly simple task to drop the leftmost label(s) so as to 
obtain the mail domain, but doing that properly requires a zone cut 
algorithm that most servers miss.

> follow the existing accepted standard of using a valid resolvable FQDN 
> in EHLO.

If we have a weaker goal and an admirable one, we're better off if 
they don't conflict with each other. We cannot ask for a domain name 
after EHLO, except for tiny ESPs whose domain name, host name, and IP 
address are the same thing. If we need the domain name, we can either 
mandate the zone cut algorithm, or use a different verb than EHLO.

---

From: Paul Russell <[email-masked]>
Date: Wed, 17 Jun 2009 09:59:03 -0400

> Sure, it is the the be strict in what you send, lenient in what you receive.

The first part of that has always been a good idea.  The second part may have
been workable ten years ago, but on today's Internet, exercising leniency in
what you accept will get you flooded with all sorts of traffic that you neither
want nor need.

Ten years ago, we accepted inbound mail from systems with missing or mismatched
DNS information and from systems which used a non-resolvable hostname as their
HELO/EHLO name.  Today, we reject all mail from these systems.

-- 
Paul Russell, Senior Systems Administrator
OIT Messaging Services Team
University of Notre Dame

---

From: Jeff Macdonald <[email-masked]>
Date: Wed, 17 Jun 2009 10:28:06 -0400

On Wed, Jun 17, 2009 at 6:45 AM, Ian Eiloart<[email-masked]> wrote:
<snip>
> It might not be too late for some leadership initiative to

+1

-- 
Jeff Macdonald
Ayer, MA

---

From: Steve Atkins <[email-masked]>
Date: Wed, 17 Jun 2009 07:59:54 -0700

On Jun 16, 2009, at 4:17 PM, Daniel Feenberg wrote:
>  Because it would be impossible to maintain a DNSBL for IPV6,

I keep hearing people say this, but I've not seen any clear  
justification for it. It seems to me to be no more difficult to run a  
blacklist for IPv6 addresses than IPv4 addresses (neither is easy, but  
the details of the address representation don't seem to make more than  
minor differences).

Can you expand on why you think it's the case, or point me at some  
discussion of it?

Cheers,
   Steve

---

From: der Mouse <[email-masked]>
Date: Wed, 17 Jun 2009 11:03:47 -0400 (EDT)

>> [...] I think what he is actually saying is that if you have a
>> machine that says "EHLO some.name" then there should be both a MX
>> record for some.name and a SRV record for _client._smtp.some.name
>> (i.e. a CSV/CSA record).

Not quite.  It requires that the HELO/EHLO argument be a valid name for
the SMTP client host.  The presence or absence of any DNS zone cuts in
the vicinity is completely irrelevant.

> algorithm that most servers miss.

...and which is wrong anyway.  The division of DNS names into "hosts"
and "domains" is purely a human one.  Dropping the first label from a
DNS name in an attempt to get "the domain" for it is, at best, a rough
heuristic.  Looking up the DNS tree for zone cuts also is nothing more
than a heuristic.

It's not even clear to me that there *is* a "_the_ domain".  What's
"the domain" for (to invent an example) mail.research.tjw.ibm.com?
There plausibly could be as many zone cuts as there are dots, there,
and I could argue for picking any of them as "the domain" for email
responsibility purposes (well, possibly excepting the TLD, but even
that is just a heuristic, likely to break soon).

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: Daniel Feenberg <[email-masked]>
Date: Wed, 17 Jun 2009 11:17:24 -0400 (EDT)

On Wed, 17 Jun 2009, Steve Atkins wrote:

> I keep hearing people say this, but I've not seen any clear justification for 
> it. It seems to me to be no more difficult to run a blacklist for IPv6 
> addresses than IPv4 addresses (neither is easy, but the details of the 
> address representation don't seem to make more than minor differences).
>
> Can you expand on why you think it's the case, or point me at some discussion 
> of it?

Of course a spammer could reuse an IPV6 address, and then a DNSBL could 
catch subsequent spam from that address. But there isn't any need to reuse 
IPV6 addresses - they are nearly infinite in number, each customer is 
assigned billions by default and there is no real need for the spammer to 
restrict himself to his officially listed addresses.

IPV4 DNSBL work, even though they are "listing badness" because IPV4 
address space is finite. That means that "listing badness" isn't really 
different from "listing goodness". But if badness is infinite, then 
listing bad addresses won't be effective.

Note that my argument that MTAs with only IPV6 won't be established is not 
contradicted by the existence of MTAs with IPV6 and IPV4 connectivity. Nor 
does it really depend on the difficulties with DNSBLs, although that is an 
additional obstacle. The major obstacle is the limited connectivity that 
an IPV6 only MTA would have.

Daniel Feenberg

---

From: der Mouse <[email-masked]>
Date: Wed, 17 Jun 2009 11:19:30 -0400 (EDT)

> Can you expand on why you think it's the case, or point me at some

I haven't claimed that, but I think the idea is that the size of the v6
address space, and the amount of address space that even a tiny
customer gets, make it useless to try to identify and block bad-actor
addresses, because they'll just dodge to unblocked addresses.

There is some truth to this, but it's not that simple.  DNSBLs like the
CBL, where the assignee of the IP address is not actively malicious and
thus won't be ducking out from under the listing, will be unaffected.
There are doubtless some DNSBLs which will be unable to do anything
useful in v6, but there are others that will simply start listing /64s
or /48s or some such.

It will change things, definitely.  But to say it will be impossible to
maintain a DNSBL is an overstatement.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: Steve Atkins <[email-masked]>
Date: Wed, 17 Jun 2009 08:35:17 -0700

On Jun 17, 2009, at 8:17 AM, Daniel Feenberg wrote:

>> justification for it. It seems to me to be no more difficult to run  
>> a blacklist for IPv6 addresses than IPv4 addresses (neither is  
>> easy, but the details of the address representation don't seem to  
>> make more than minor differences).
>>
> Of course a spammer could reuse an IPV6 address, and then a DNSBL  
> could catch subsequent spam from that address. But there isn't any  
> need to reuse IPV6 addresses - they are nearly infinite in number,  
> each customer is assigned billions by default and there is no real  
> need for the spammer to restrict himself to his officially listed  
> addresses.

Which is why you'd list the /64 or /48 in most cases. That's not  
difficult to do, even with bind, and is easy to manage with any decent  
database backend.

> address space is finite. That means that "listing badness" isn't  
> really different from "listing goodness". But if badness is  
> infinite, then listing bad addresses won't be effective.

I don't think that reasoning really holds water, there. IPv6 space is  
also finite. There'd need to be minor operational changes to support  
it, and there are a couple of naive approaches currently used in IPv4  
that would fail dismally in IPv6 without some changes, but nothing  
particularly difficult.

If anything, reduction in use of NATs might make some sorts of  
blacklist more accurate and effective in IPv6 space than in IPv4.

> Note that my argument that MTAs with only IPV6 won't be established  
> is not contradicted by the existence of MTAs with IPV6 and IPV4  
> connectivity. Nor does it really depend on the difficulties with  
> DNSBLs, although that is an additional obstacle. The major obstacle  
> is the limited connectivity that an IPV6 only MTA would have.

Cheers,
   Steve

---

From: Lyndon Nerenberg <[email-masked]>
Date: Wed, 17 Jun 2009 10:21:17 -0600

> what you accept will get you flooded with all sorts of traffic that

"... be lenient in what you receive" *never* meant you should accept and
attempt to second-guess the intent of anything in violation of the
applicable protocol spec. It meant "don't tip over and die when
presented with crap." I.e., fail gracefully.

This was very important on things like the IMPs when the protocols and
their implementations were immature. It's still important today, but
only when interpreted with its original meaning and intent.

--lyndon

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 17 Jun 2009 18:50:44 +0200

> the vicinity is completely irrelevant.

Isn't the FQDN for a host the host name "dot" the domain name?

> heuristic.  Looking up the DNS tree for zone cuts also is nothing more

The host gets its name after some buddy edits the zone file. Which 
zone file? The domain's one. Yes, it is human, heuristic, and error 
prone. (I never seriously meant to actually implement a zone cut 
algorithm in MTA servers in order to derive domain names. However, 
that was an early hypothesis for the SPF check algorithm, as an 
alternative to requiring SPF records for each possible helo name.)

> "the domain" for (to invent an example) mail.research.tjw.ibm.com?

If research.tjw.ibm.com had an MX, it would be a good candidate. 
Otherwise... elementary my dear Watson. Is that worse than Bayesian 
guesses?

> that is just a heuristic, likely to break soon).

Yeah, John recently wrote something about .va sporting an MX (John 
Levine, not john.vatican.va) while 2nd level co.uk has none. It is 
much better if the domain is plainly told by the client rather than 
badly guessed by the server. E.g. "VHLO domain.name".

---

From: Douglas Otis <[email-masked]>
Date: Wed, 17 Jun 2009 10:25:49 -0700

On Jun 16, 2009, at 6:55 PM, Lyndon Nerenberg wrote:

>> IMHO, all outbound MTAs should be required to return CVS records  
>> for their EHLO name and offer MX records for their inbound.
>
> Doug, are you sure that's what you meant to say? The sentence is a  
> bit ambiguous. Are you really saying any host that sends mail (is an  
> SMTP client) MUST also host an listed SMTP server?

You are right.  This should have been pubic MTAs, meaning those that  
send email using port 25 without authentication.  All other sources  
should be restricted in some fashion, perhaps by ACLs or  
authentications.

-Doug

---

From: "J.D. Falk" <[email-masked]>
Date: Wed, 17 Jun 2009 11:52:50 -0600

> hostname of adsl-76-203-192-33.dsl.rcsntx.sbcglobal.net.

That list is based on what we've observed recently in our data network, 
collating delivery logs from a whole bunch of ISPs and other sources.  Since 
it's aggregating multiple views, we tend to see more than a single site would.

(You can look up some data without registering, but not all.  Either way, 
it's still free.)

-- 
J.D. Falk
Return Path Inc
http://www.returnpath.net/

---

From: John Levine <[email-masked]>
Date: 17 Jun 2009 17:53:32 -0000

>Isn't the FQDN for a host the host name "dot" the domain name?

The FQDN for a host is the host's FQDN.  As we've all noted, there's
lots of heuristics to guess domain names, none of which work.

R's,
John

---

From: John Levine <[email-masked]>
Date: 17 Jun 2009 17:55:34 -0000

>Because it would be impossible to maintain a DNSBL for IPV6

Why?  I expect that v6 DNSBLs will list /64s by default, which makes
them roughly analogous to v4 DNSBLs.

I also doubt that it is a very good idea to assume that DNSBLs will
always be the way that people do reputation management.

R's,
John

---

From: Dotzero <[email-masked]>
Date: Wed, 17 Jun 2009 13:59:53 -0400

> (You can look up some data without registering, but not all.  Either way,

The reason I said to register is that the information available to
non-registered users would not have helped answer the question he
posed.

---

From: der Mouse <[email-masked]>
Date: Wed, 17 Jun 2009 14:14:32 -0400 (EDT)

>> [SMTP] requires that the HELO/EHLO argument be a valid name for the
>> SMTP client host.

Only if you define "host name" and "domain name" that way.  It would be
logically consistent to do so (though somewhat confusing, because "host
name" and "domain name" are already in wide use to mean other things).

Defining them that way is really just giving names to a heuristic.  The
"domain name" determined that way will be what you want for a
substantial fraction of your mail clients, but by no means all (well,
unless you have a remarkably unusual mail stream).

> zone file?  The domain's one.

That amounts to defining a host's "domain name" to be the next zone cut
at-or-above it in the DNS hierarchy.  That's a consistent definition,
and useful as a heuristic, but will fail in enough cases it's no better
than that.  (Fail meaning give you something other than what you want.)

Actually, it amounts to that for most hosts; for glue nameservers, the
"which zone file" heuristic gives a different domain than the "next
zone cut" one.  But neither one is better than a heuristic.  (Glue
nameservers usually aren't mailservers for mid- to large-size sites,
but for small private sites, it's not uncommon for everything to be on
the same machine.)

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: Douglas Otis <[email-masked]>
Date: Wed, 17 Jun 2009 17:18:32 -0700

On Jun 17, 2009, at 7:59 AM, Steve Atkins wrote:
> a blacklist for IPv6 addresses than IPv4 addresses (neither is easy,  
> but the details of the address representation don't seem to make  
> more than minor differences).
>

Factors that make managing an IPv6 block-listing service fairly  
impractical go beyond 96 additional address bits.  The term  
impractical is used because, with enough money, anything is possible.

1) Publishing behavior based address lists require evidence collection  
in the event of disputes, and this does not scale well. (expensive)

2) IPv6 to IPv4 and IPv6 to IPv6 NATs obfuscates who might be  
involved. (problematic)

3) Reverse DNS scanning does not scale well. (slow)

4) Diverse and rapidly expanding address space allows bad-actor's  
activity to stay ahead of the massive amounts of IP address related  
information publishing.  (futile)

5) An extremely low cost for IP addresses allows bad actors to persist  
at sporadic use for many years. (futile)

The collection of evidence is often constrained by the related  
identifier, such as the IP address.  Unfortunately, IPv6 allows a new  
IP address to be used for each message sent.  Some countries already  
place thousands of MTAs behind carrier grade NATs.  Any scheme that  
concludes the use or abuse of an IP address indicates which individual  
originated a message makes it ill-equipped to deal with today's  
evolving network environment.  Block-listing must soon be replaced by  
acceptance-listing.  Things like CSV would permit a means to  
consolidate the sources managed by acceptance-lists, where reputation  
could be based upon MTA domains, rather than a vast array of IP  
address or the domains controlled by their customers.

With email so heavily abused, attempting to acquire and process the  
ten to two hundred SPF transactions per message is not possible at  
today's magnitude of abuse per legitimate message.  SPFs use of macros  
and exists functions requires a process done in conjunction with  
message acceptance, while at the same time placing DNS itself in  
peril.  Any strategy that attempts to place the customers of an email  
provider responsible simply does not scale as an abuse deterrent.   
Management efforts can be reduced a million fold when directly holding  
the immediate outbound public MTA accountable, rather than attempting  
to shift accountability toward one of a provider's many users, which  
is the real (and admitted) motivation behind SPF/Sender-ID/A-R.  This  
Nast cartoon depicts the situation fairly well by changing the  
question to asking who is responsible for email's dire situation and  
having the men in the circle represent that of the providers.

http://en.wikipedia.org/wiki/File:Tammany_Ring,_Nast.jpg

Pushing responsibility to the edge does not work, and email provides  
ample evidence.  This nonsense is putting the Internet itself in peril.

-Doug

---

From: der Mouse <[email-masked]>
Date: Wed, 17 Jun 2009 20:51:13 -0400 (EDT)

> impractical go beyond 96 additional address bits.

> 1) Publishing behavior based address lists require evidence
> collection in the event of disputes, and this does not scale well.
> (expensive)

I can't see why v6 versus v4 makes any difference at all to this.

> 2) IPv6 to IPv4 and IPv6 to IPv6 NATs obfuscates who might be

I can't see why this is any worse than the v4-to-v4 NATs the net is
already full of.

> 3) Reverse DNS scanning does not scale well. (slow)

True.  DNSBLs that depend on rDNS scanning may die.  There are plenty
of DNSBLs, including some of the most useful, that do not.

> information publishing.  (futile)

I see no real difference here between a v4 list that lists at the /32
level and a v6 list that lists at the /48 (or maybe even /64) level.

> 5) An extremely low cost for IP addresses allows bad actors to
> persist at sporadic use for many years.  (futile)

And this differs from v4...how?

> IP address to be used for each message sent.

So, collect evidence at the /64, or even /48, level, rather than at the
individual address level.

Even to the extent that these problems are real, they are theoretical.
It certainly behooves us to think about them ahead of time, but absent
experience demonstrating that they are more than potential, I don't see
them as a reason to give up on v6 DNSBLs without even trying.

> ample evidence.

It's not that doing that has been tried and found wanting; rather, it
has not been tried.  (Actually, it has been tried in a limited way;
there are pieces of the net that _do_ push responsibility to the end
user.  Oddly enough, they are basically nonexistent as far as abuse
emitters go; what evidence I see indicates that it _does_ work.)

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: Ian Eiloart <[email-masked]>
Date: Thu, 18 Jun 2009 10:39:48 +0100

Oddly, most of this thread isn't really relevant to the subject line.

The reason for asking the question is that we don't currently have widely 
published SPF records. If we did, then we wouldn't need IP address 
reputation for either IPV4 or IPV6. We'd just need the answer to two 
questions:

1. Does that IP address have permission to emit mail from the sender domain?
2. Do I want email from that sender? If the sender doesn't have a good 
reputation, perhaps the domain does? If the reputation is neutral, I'll 
take a good look at the content and the delivery rate.

Even the above isn't relevant to the OP, who just wanted to diagnose a 
delivery failure - but that is likely to have been a symptom of some 
anti-spam measure that wasn't domain reputation related.

---

From: Douglas Otis <[email-masked]>
Date: Thu, 18 Jun 2009 12:00:25 -0700

> I can't see why v6 versus v4 makes any difference at all to this.

You have concluded that resolution of an IPv6 address reputation can  
safely ignore portions of the address.  Assuming the lower bits don't  
matter (64 bits of interface or 16 bits of site level identifiers,  
which will in some cases), that still leaves 53 or 37 bits of unicast  
address space in which to contend.

This also represents a space likely to evolve as routes become  
consolidated.  In an effort to constrain zone growth and to thwart  
address hopping, IPv4 addresses may have been consolidated into CIDRs  
that don't cross route announcements when the majority of addresses  
abuse email.  This might then result in a CIDR of /30 that contains a  
single IP address where evidence had not been collected.  Invariably,  
the user of that one IP address will complain and request removal.   
Imagine who might be affected when CIDR consolidation goes from groups  
of of 256 or 4096 to groups of 18x10^18 or 1.2x10^24 based upon the  
evidence of a single IP address?

In addition, the collection of IP address related evidence must be  
retained and reviewed.  IPv6's increased range ensures a greater  
workload for review and a much greater need for storage.  This  
increase will entail a sizable expenditure.

> I can't see why this is any worse than the v4-to-v4 NATs the net is  

It is not as common to find carrier grade NATs.  Excluding these  
sources has already become problematic in a few countries, and will  
only get worse.

> True.  DNSBLs that depend on rDNS scanning may die.  There are  
> plenty of DNSBLs, including some of the most useful, that do not.

A reactive system that list addresses as abused and then automatically  
expire provides bad-actors two advantages:

a) bad-actors can avoid effective blocking by rapidly moving both  
source and target.

b) professional bad-actors can quickly identify the location of spam  
traps.

> I see no real difference here between a v4 list that lists at the / 
> 32 level and a v6 list that lists at the /48 (or maybe even /64)  
> level.

It already takes minutes to build zones and distribute information.   
Making zones larger increases processing and transfer time which  
already erodes effectiveness.

> And this differs from v4...how?

It is fairly common to see abusive activity cycle through a range of  
addresses for one day every few weeks.  With IPv6, the addresses being  
abused could then cycle through a range for 1 minute every decade.   
What policy deal effectively with that strategy?

>> identifier, such as the IP address.  Unfortunately, IPv6 allows a  
>> new IP address to be used for each message sent.
>
> So, collect evidence at the /64, or even /48, level, rather than at  
> the individual address level.
>
> Even to the extent that these problems are real, they are  
> theoretical. It certainly behooves us to think about them ahead of  
> time, but absent experience demonstrating that they are more than  
> potential, I don't see them as a reason to give up on v6 DNSBLs  
> without even trying.

It seems insane to repeatedly do the same thing and then expect  
different results each time.  IPv4 is already approaching the majority  
of all the addresses being blocked.  It will not be long for this to  
transition into only listing IP addresses registered as outbound mail  
servers.  The registration process might be as simple as having both  
an MX and CSV record published.

>> Pushing responsibility to the edge does not work, and email  
>> provides ample evidence.
>
> It's not that doing that has been tried and found wanting; rather,  
> it has not been tried.

Have you heard of SPF?

SPF represents a strategy to shift MTA accountability onto the domains  
of their customers (the preverbal edge).  This scheme may entail a  
maximum of 10 or 11 SPF record transactions, which may then include 10  
transactions per contained MX target.  For each legitimate message,  
there is typically at least 20 abusive messages (which also publish  
SPF records).  So the potential of 10 to 200 additional SPF  
transactions (when Sender-ID is included) should be multiplied by 20,  
where the 200 transactions becomes 4000 per legitimate message.

The ratio of legitimate to abusive is getting worse.  At what point  
does one admit that SPF, when used as designed, does not scale, that  
it imperils the integrity of DNS, and that it imperils the viability  
of SMTP.  If nothing else, SPF/Sender-ID has been responsible for  
delaying more scalable and safer solutions.  Just as you can't ask  
lenders using securitization to dodge lending accountability how they  
might be held accountable, you can't ask a group of large providers  
this question either and expect a reasonable answer.  The system needs  
to hold providers directly accountable.  That is what CSV attempted,  
where publishing too many such records or using too many EHLOs should  
also be considered abusive.

> (Actually, it has been tried in a limited way; there are pieces of  
> the net that _do_ push responsibility to the end user.  Oddly  
> enough, they are basically nonexistent as far as abuse emitters go;  
> what evidence I see indicates that it _does_ work.)

Can you provide some specifics?

-Doug

---

From: der Mouse <[email-masked]>
Date: Thu, 18 Jun 2009 15:19:24 -0400 (EDT)

> IPv4 is already approaching the majority of all the addresses being
> blocked.

Well...not all that closely, at least not by people who care about
their mail.

The Spamhaus Zen list, arguably one of the most useful, best run, and
least fringe of DNSBLs, currently lists slightly over 521 million
addresses, a touch under 1/8th of the IPv4 space.  Removing the PBL
component - meaning, just bad actors (for SBL-XBL defintiions of
"bad"), not including addresses whose providers say they're not
supposed to be sending mail but which haven't necessarily been observed
doing so - brings this down to between 11 and 12 million addresses,
which is tiny; it's between 1/4 and 1/3 of 1% of IPv4 space.

>> It's not that doing that has been tried and found wanting; rather,

Yes.  It flopped, no?  If it had been widely adopted, I might have to
decide whether I think it constitutes "pushing responsibility to the
edge".  But I don't.

> Can you provide some specifics?

I worked for McGill (a university in Montreal) for most of my career,
some 15 years, about 10 of those as postmaster for one of the labs
there.  Certainly we, and to the extent that I saw it the rest of the
University, imposed responsibility on end users.

I am currently working for Openface Internet, an ISP in Montreal.
While it's done very differently (the provider/user relationships are
vastly different in the two contexts), we too push responsibility for
the user of the address space we grant our customers to those
customers.

Neither of these were completely abuse-free.  But nobody their size is
- heck, even my house /28 once emitted some misbehaviour for a few
minutes, when someone asked my help getting a machine cleaned up and I
made a mistake with filtering - and I've seen no reason to think they
were/are even slightly above the background noise.  (If anyone has
evidence to the contrary for Openface, please tell me; I'll certainly
get it into the hands of our abuse handlers, and can track the issue.
For McGill, I'm less able to act since I no longer work there, but it
hasn't been so long I don't still know people.)

Yes, they're tiny fractions of the net.  Openface has a /19 and a /20.
McGill had two /16s - I don't know what they have now.  That doesn't
invalidate my point.

Other places, where similar policies and enforcement exist (or have
existed), have exhibited similarly low abuse-emitter profiles in my
experience.  Other Montreal universities.  RCN while Afterburner was
abuse lead there - that was during a period when I made a point of
complaining about every spam that got through to me, spending hours a
day writing spam complaints, and I had all of _one_ occasion to write
to RCN, which drew prompt and effective response.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: Rich Kulawiec <[email-masked]>
Date: Thu, 18 Jun 2009 16:46:33 -0400

> addresses to their legitimate behaviors in a generalized model would be a 

+1.  For instance, I (semi-seriously, semi-facetiously) proposed "XM"
records some years ago, whose value would be 0 or 1: hosts with 1 send
SMTP traffic, hosts with 0 don't.  Thus every MX's behavior could be
to reject all port 25 SMTP connections from hosts with XM=0.

There a lot of problems with this idea, and if memory serves, both
Dave Crocker and John Levine pointed them out at the time.  But I think
that perhaps it's time to revisit the general concept and see if it
could be made to work for (as Bill said) "legitimate behaviors in a
generalized model".  This would not only allow us to address SMTP abuse,
but (for example) zombie hosting of DNS and HTTP servers.

---Rsk

---

From: Rich Kulawiec <[email-masked]>
Date: Thu, 18 Jun 2009 16:47:11 -0400

> what you accept will get you flooded with all sorts of traffic that you neither

I strongly agree with Paul on this point.  I'd *like* to be lenient about
what I permit systems to receive, but circumstances dictate otherwise.
We no longer have an Internet where good behavior is the norm and where
system/network operators make that their primary goal; we have an Internet
where abuse is the norm and most system/network operators not only don't
care about good behavior, but have the audacity to complain when anyone
points out that they're not exhibiting it.

---Rsk

---

From: Daniel Feenberg <[email-masked]>
Date: Thu, 18 Jun 2009 17:22:16 -0400 (EDT)

> Dave Crocker and John Levine pointed them out at the time.  But I think

Are there problems that would extend beyond the problems of traders in 
improper material who don't want their material sitting in queues on the 
ISP MTA? This is usually dressed up as "The FBI is after me for my 
advanced political views" or "My ISP is an evil monopolist", but are there 
problems for other users of email?

I would also add that the "end to end" principle, however much it applies 
to voluntary associations between endpoints, can hardly be applied to the 
SMTP protocol, where complete strangers are expected to interact. Sites 
will always be cautious of strangers, and asking that SMTP senders be 
vouched for by their DNS providor is a very small concession indeed.

Furthermore, the "endpoint" in the "end-to-end" principle is a host, not
a user, so it is perfectly within the principle to is a host IP address as 
a discrimination device.

Daniel Feenberg

---

From: John Levine <[email-masked]>
Date: 18 Jun 2009 21:42:21 -0000

>> [ marking valid SMTP client hosts ]
>ISP MTA?

The problems were technical, putting records other than PTR in the
rDNS zone.  That's why Dave and I came up with CSV.

Pretty please, read http://www.mipassoc.org/csv/ if you want to continue
this discussion.

R's,
John

---

From: der Mouse <[email-masked]>
Date: Thu, 18 Jun 2009 17:57:03 -0400 (EDT)

> Pretty please, read http://www.mipassoc.org/csv/ if you want to
> continue this discussion.

Offhand, I don't see anything there that explains what's wrong with
putting records other than PTR under an rDNS zone.  Certainly _some_
non-PTR record types cause no problems, such as CNAME and zone cut
administrative records like SOA and NS.  In short, I don't see what's
wrong with XM from a technical standpoint.  What am I missing?

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: Douglas Otis <[email-masked]>
Date: Thu, 18 Jun 2009 16:01:36 -0700

On Jun 18, 2009, at 12:19 PM, der Mouse wrote:

>> Have you heard of SPF?
>
> Yes.  It flopped, no?  If it had been widely adopted, I might have  
> to decide whether I think it constitutes "pushing responsibility to  
> the edge".  But I don't.

The answer to this question depends upon who is asked.  MTA vendors  
are further enabling use of this practice, often only offering an  
impression of security.  Avoiding accountability remains the siren  
song luring large providers.

>>> enough, they are basically nonexistent as far as abuse emitters  
>>> go; what evidence I see indicates that it _does_ work.)
>>
> I worked for McGill (a university in Montreal) for most of my  
> career, some 15 years, about 10 of those as postmaster for one of  
> the labs there.  Certainly we, and to the extent that I saw it the  
> rest of the University, imposed responsibility on end users.

This control is "out-of-band" from the abused protocol, and not the  
result of all recipients of the protocol resolving possible identities  
of each of university users.

> While it's done very differently (the provider/user relationships  
> are vastly different in the two contexts), we too push  
> responsibility for the user of the address space we grant our  
> customers to those customers.

The point attempted was to ensure providers are held accountable for  
their stewardship at handling abuse.

Schemes that pass accountability onto what might be feckless domain  
owners are inherently evil.  With SPF/Sender-ID, providers take the  
persona of their purported customer's domain.  Those describing SPF  
often erroneously suggest MTA authorization provides evidence a  
message originated from those providing the authorization.  Providers  
regulating their users is not what is being advocated by those  
promoting SPF/Sender-ID and advocating moving accountability to the  
edge.  Their desire is to have end recipients conducting many  
transactions to reveal an authorization, but not the one transaction  
that could have revealed the identity of the provider.  As such,  
providers can remain lax in their stewardship, and the end-point  
victim is expected to preform an ever growing number of transactions  
as the quantity of junk emails rise.

Providers MUST be held _directly_ accountable.  The financial debacle  
occurred when lenders avoided accountability by securitizing their  
service.  Avoidance of accountability provides evidence of regulation  
failure.  While service providers may advocate various schemes to  
avoid accountability, such schemes are doomed.  When they create  
innocent victims, they are evil.

-Doug

---

From: Douglas Otis <[email-masked]>
Date: Thu, 18 Jun 2009 16:28:35 -0700

> rDNS zone.  That's why Dave and I came up with CSV.

Resolution failures while attempting to obtain a reverse DNS record  
consumes a fair amount of server resources due to the high levels of  
abuse.

-Doug

---

From: der Mouse <[email-masked]>
Date: Thu, 18 Jun 2009 21:29:09 -0400 (EDT)

>> I worked for McGill [...]
> result of all recipients of the protocol resolving possible
> identities of each of university users.

Both true.  So?

Responsibility, in the sense of accountability for (potential) abuse,
is a meatspace thing, not amentable to being part of a network
protocol, so at least _some_ of this must be done out-of-band with
respect to the protocol.

> owners are inherently evil.

I disagree, _provided_ accountability is actually passed on.  What you
appear to be thinking of is not accountability but mere identification
(albeit moderately strong identification).  That there is no real
accountability is the major fundamental problem I see with today's
Internet: domain holders are not accountable to their registrar or, in
most cases, TLD admins for what they do with their domains; address
space assignees are not accountable to their RIRs for what they do with
their address space (except for the most trivial adminstrative aspects,
such as how thorougly they're using the space assigned, and even that
not very much); email address holders on the top few webmail systems
are not held accountable by the webmail provider for how they use their
accounts.

Schemes that pass accountability on would be good.  So far, I haven't
seen any; the most I've seen is schemes that provide strong enough
authentication to make it possible to construct systems that pass
accountability on.  Nobody ever seems to take the additional step of
actually doing so.  (Well, except on a trivial scale, such as my
personal blocking of Yahoo, imposing a penalty for - ie, holding them
to account for - the abuses they don't rein in.)

> Providers MUST be held _directly_ accountable.

Right.  But until this is fixed at the top, I see little hope it will
happen in the lower levels, except sporadically.  (The places that do
do it are exceptional, and, in the cases where I'm in a position to
know why they do it, they do it not because they are held accountable
by whoever assigned the resources to them but because they are ethical
enough to feel a compulsion to do what's right even when they're _not_
overtly held accountable.  While this mindset is common enough for us
to have words for it, it is not nearly common enough to save the net
from the disasters that governmental disconnect between authority and
responsibility leads to.)

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: Bill Cole <[email-masked]>
Date: Thu, 18 Jun 2009 22:43:32 -0400

> Sure, it is the the be strict in what you send, lenient in what you

Yes, and with all due respect to Jon Postel, that principle is better suited 
to the Internet of the 1980's than it is to the Internet today. When there 
were a small number of ultimate authorities who put a de facto ceiling on 
leniency and the net was treated as a carefully watched experiment rather 
than as a production utility,

> We could say something like all emails on IPv6 must have a DKIM
> signature, have RDNS helo, etc... as there is not much of an
> implementation with IPv6, there is a chance for these practices to be
> adopted from day one...

As John pointed out, "Day One" is in the past and the IETF is hostile *as a 
matter of principle* to defining application layers differently for IPv6.

I don't think that principle is wrong. Also, I can't think of many examples 
of RFC's successfully performing the role you describe of leading a 
significant change in practice rather than describing what is already widely 
being done to good effect.

Put more directly: if you want something to end up as de facto mandatory and 
have a chance of ending up in an RFC, you have to start by getting it 
enforced by many working production systems.

> From: "Bill Cole"<[email-masked]>
> To: "Anti-Spam Research Group - IRTF"<[email-masked]>
> Sent: Tuesday, 16 June, 2009 10:14:02 PM GMT +01:00
>    Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna
>> Knowing that mail servers are not deployed on IPv6, what would it take
>> to make all these requirements mandatory for IPv6 and start with a
>> better infrastructure than on IPv4?
>
> they never bothered to read it. That is possible because the major MTA's

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 19 Jun 2009 08:41:57 +0200

> protocol, so at least _some_ of this must be done out-of-band with

On thefreedictionary I read
   Synonyms:  responsible, answerable, liable, accountable, amenable
   These adjectives share the meaning obliged to answer, as for one's
   actions, to an authority that may impose a penalty for failure.

Because the IETF cannot even enforce protocol compliance, addressing 
responsibility implies identifying an authority that has the power 
of imposing some kind of penalty.

> responsibility leads to.)

I think we can safely withdraw the naive picture where carriers act 
as authorities, and forget about the possibility that anything will 
be eventually "fixed at the top", except for possible devout 
beliefs. On this Earth, ethical mindsets are still powerful 
intellectual tools that bring visions and may allow to plan for 
decades. Although such planning usually results in optimization of 
revenues in the long run, uncertainty about the future wreaks those 
greedy and short-sighted behaviors that currently are the norm.

To cope with that, protocols need to introduce ad-hoc authorities 
whenever responsibility is required. For mail, those may involve 
DNSBLs, CAs, VBR vouchers, and similar kinds of independent 
organizations. We are already relying on them, unofficially. For 
increased cooperation, we better make that explicit.

---

From: Ian Eiloart <[email-masked]>
Date: Fri, 19 Jun 2009 09:19:22 +0100

--On 18 June 2009 22:43:32 -0400 Bill Cole <[email-masked]> 
wrote:

>
> I don't think that principle is wrong. Also, I can't think of many
> examples of RFC's successfully performing the role you describe of
> leading a significant change in practice rather than describing what is
> already widely being done to good effect.

Which is why I said it has to be done the other way around. Some 
organisation with a significant email user base needs to take a lead on 
this. It could be a large ISP, a large webmail provider, a government, or 
some other body. It has to be done before the situation gets out of hand, 
though.

---

From: Ian Eiloart <[email-masked]>
Date: Fri, 19 Jun 2009 09:52:16 +0100

> decide whether I think it constitutes "pushing responsibility to the

No, it hasn't flopped. It just hasn't reached critical mass yet.
<http://www.openspf.org/Statistics>  has some interesting stuff, including 
reports by Microsoft of their experience of SenderID, from Google of their 
positive experience with SPF and DKIM.

These surveys show the rate of growth of SPF record publication:

<http://dns.measurement-factory.com/surveys/>
2005 - not measured
2006 - 6% of 1,756,827 zones with at least one working nameserver published 
SPF records
2007 - 12.6% of 2,053,150 ''
2008 - 16.72% of 1,000,000 ''

Google's conclusion: "Reputation based upon the authenticated sender’s 
domain works well for us. Spammers are easily identiﬁed, as are good 
senders. Only a few senders end up in the grey area in between, and for 
these we still have the traditional statistical filtering methods for 
classification. It is not without a few problems though, but we think they 
are small enough that they can be worked out eventually. We think 
reputation systems can work well for others too, and we hope to see more 
senders authenticating and observing good sending practices because of it."

They're referring to a combination of SPF (good for early decisions, based 
on recipient preferences) and DKIM (protects forwarded email).

---

From: Ian Eiloart <[email-masked]>
Date: Fri, 19 Jun 2009 09:55:12 +0100

--On 19 June 2009 08:41:57 +0200 Alessandro Vesely <[email-masked]> wrote:

> responsibility implies identifying an authority that has the power of
> imposing some kind of penalty.

It's our respective governments, and such international law as they agree 
to.

As an internet community, we must work towards forcing people to 
authenticate senders - by making it harder and harder for unauthenticated 
mail to get delivered. Once we know who's sending the email, they can be 
held to account by (a) reputation services, and (b) the law.

> I think we can safely withdraw the naive picture where carriers act as
> authorities, and forget about the possibility that anything will be
> eventually "fixed at the top", except for possible devout beliefs. On
> this Earth, ethical mindsets are still powerful intellectual tools that
> bring visions and may allow to plan for decades. Although such planning
> usually results in optimization of revenues in the long run, uncertainty
> about the future wreaks those greedy and short-sighted behaviors that
> currently are the norm.
>
> whenever responsibility is required. For mail, those may involve DNSBLs,
> CAs, VBR vouchers, and similar kinds of independent organizations. We are
> already relying on them, unofficially. For increased cooperation, we
> better make that explicit.

---

From: John Levine <[email-masked]>
Date: 19 Jun 2009 09:43:26 -0000

>wrong with XM from a technical standpoint.  What am I missing?

As I recall, it was about systems that "know" that there'll never
be anything but PTR, zone cuts, and maybe CNAME in the rDNS.  In
theory it could be fixed, but as we know the difference betweeen
theory and practice is greater in practice than it is in theory.

R's,
John

---

From: John Levine <[email-masked]>
Date: 19 Jun 2009 10:06:04 -0000

>   actions, to an authority that may impose a penalty for failure.

I think that as soon as you start quoting the dictionary, you've lost
the argument.

In any technical field, one needs jargon as a shorthand to refer to
agreed concepts.  Some jargon words are made up (byte), but most are
normal words used in new ways (parser).  When you start quoting the
dictionary, you're arguing that some word has only its conventional
meaning, which is generally not true for any word used in a technical
context.  Words like responsible, accredit, certify, are evolving into
jargon, but we haven't yet come to consensus on what if any jargon
meaning they have.  The dictionary doesn't help, so leave it out of
this discussion, please.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 19 Jun 2009 12:54:17 +0200

> lots of heuristics to guess domain names, none of which work.

What about the other way around: given a domain and an IP address, can 
we say whether the IP address "is a member of" the domain?

Vhlo mentions the following three ways to determine that, without 
apparently resorting to heuristics. I'm wondering how sound it is to 
rely on those, or similar, techniques.

* rDNS returns a name whose right part matches the domain name,
* an MX record for the domain mentions a host with the given IP,
* the IP address passes the SPF check for that domain.

---

From: David Wilson <[email-masked]>
Date: Fri, 19 Jun 2009 12:14:20 +0100

> we say whether the IP address "is a member of" the domain?

I would ask: what do you mean by 'domain' in this context?

(The term domain is probably one of the most over-used terms.)

In the context of email addresses, the domain is the part of the address
to the right of the '@'.

I'm not sure if there is a definition for 'domain' as it applies to
hosts and their domain names (fully or partly qualified). There is a
local (to a resolver) definition for searching which can convert a PQDN
to a FQDN, but that is not global data.

There is, of course, the zone in which the FQDN is defined for use in
the DNS.

regards

David

---

From: der Mouse <[email-masked]>
Date: Fri, 19 Jun 2009 08:17:25 -0400 (EDT)

> What about the other way around: given a domain and an IP address,
> can we say whether the IP address "is a member of" the domain?

We can.  We can also say the IP address and the domain live on the same
shelf in the supermarket, too; I'm not convinced either is a more
meaningful or useful statement than the other.

> rely on those, or similar, techniques.

If any of them results in a definition of "member of" that turns out to
be useful for whatever purpose you have in mind, sure.  I'm not sure
any of them does, but I'm also unclear on why you'd want this sort of
association between addresses and domains, so that doesn't mean much.

> * the IP address passes the SPF check for that domain.

One that's based on something designed for mail flowing to the domain;
one ditto for mail flowing from the domain; one that's based on
something not designed for mail at all.  Offhand, I'd guess that which
one is most appropriate depends on whether you're concerned with mail
flowing to the domain, mail flowing from the domain, or something other
than mail.

The things you list might be "without apparently resorting to
heuristics", and in a sense that's true, in that each one is
well-defined and has a mechanically testable definition. But that
doesn't keep them from being heuristics, depending on what you're using
them for.  Using MX records for anything other than determining what
host to connect to to deliver mail is, at best, a heuristic.  As is
each of the others, when used for anything other than its respective
purpose.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: der Mouse <[email-masked]>
Date: Fri, 19 Jun 2009 08:36:47 -0400 (EDT)

>> In short, I don't see what's wrong with XM from a technical
>> standpoint.  What am I missing?
> As I recall, it was about systems that "know" that there'll never be
> anything but PTR, zone cuts, and maybe CNAME in the rDNS.

But why does that make it useless for people whose systems _don't_ make
stupid assumptions like that?  Would it be such a bad thing to need a
non-broken DNS server to take advantage of a new feature?  Or am I
still missing something?

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: "J.D. Falk" <[email-masked]>
Date: Fri, 19 Jun 2009 11:23:42 -0600

Ian Eiloart wrote:

> this. It could be a large ISP, a large webmail provider, a government,
> or some other body. It has to be done before the situation gets out of
> hand, though.

If the ASRG were to publish some research showing that this is a good idea, 
it'd go a long way towards convincing one of those organizations you 
mentioned to consider implementing it.

-- 
J.D. Falk

---

From: Rich Kulawiec <[email-masked]>
Date: Fri, 19 Jun 2009 13:54:16 -0400

> wrong with XM from a technical standpoint.  What am I missing?

Let me see if I can find the archives of that discussion and produce
a synopsis.  (I don't want to misrepresent what John and/or Dave said
about it at the time.)

Meanwhile, let me say something about the intent -- as I had it in mind
when I brought it up.  I did so at the time that we were being increasingly
faced with zombie-originated spam, and were looking for ways to make it stop,
since clearly the irresponsible network operators hosting all those zombies
didn't treat it as an emergency requiring a whatever-it-takes committment.
(And most of them still don't, and yet have the audacity to whine when
they find their entire networks blacklisted out of exasperation that even
given YEARS to solve this urgent problem...they still haven't.)

Anyway, the point was not to make any assertion about who might be
sending mail or what domains it might be from or what might be in it:
just "this host sends mail or it doesn't".  Leaving XM=0 for acres
and acres of network space and checking for it on MX's would have the
effect of stopping zombie-direct-to-MX spam.  Of course it does nothing
for zombie-relayed-through-local-mail-system spam and nothing for all
the other nastiness zombies do: it was intended as a band-aid, no more.

Now...were I to advance this today (which I probably wouldn't given
that CSV is around) I'd suggest that it be done with forward DNS,
not reverse, and that it be checked IFF matching forward and reverse
DNS exist.  (Because if either doesn't exist or they don't match, that's
enough to reject on already.)

Anyway, let me go fish for the discussion-at-the-time and see if I can't
explain why my idea wasn't and probably still isn't a very good one.

---Rsk

---

From: Douglas Otis <[email-masked]>
Date: Fri, 19 Jun 2009 11:30:13 -0700

>> identities of each of university users.

SMTP is heavily abused, and soon IPv6 is about to become a  
necessity.   To remain practical, connectivity must be based upon  
_immediate_ and _stable_ evidence of legitimate email operation, and  
not upon any number of authorization transactions.  Each additional  
transaction to support an authorization scheme will be multiplied by  
the typical number of attempts made by abusive senders.   This means  
providers need to exclude problematic users, and not become a task  
pushed toward recipients.  Such pushing is not practical and often  
leads to unfortunate mistakes.

> Responsibility, in the sense of accountability for (potential)  
> abuse, is a meatspace thing, not amentable to being part of a  
> network protocol, so at least _some_ of this must be done out-of- 
> band with respect to the protocol.

IMHO, 99% of exclusionary practices must be handled out-of-band for  
SMTP.

> I disagree, _provided_ accountability is actually passed on.

The fact that a trusting and naive user had their domain authorize a  
provider just to have their email accepted, does not mean other  
messages emitted might not be mistaken as also belonging to that  
user's domain.  Should providers check for SPF or Sender-ID  
compliance?  How many SLA include this requirement?  When the "passing- 
on" is based upon receptions at spam traps, acceptance reliance based  
on "authorization" is likely to downgrade acceptance of the domain,  
especially when A-R headers exclude the IP address of the provider.   
Will providers really care the wrong entity had been blamed?

> What you appear to be thinking of is not accountability but mere  
> identification (albeit moderately strong identification).

Moderately strong?  Without knowing the IP address of the provider, it  
would be extremely foolish to conclude any level of identification  
assurance, especially "moderately strong".

> [...] email address holders on the top few webmail systems are not  
> held accountable by the webmail provider for how they use their  
> accounts.

Exactly.

> Schemes that pass accountability on would be good.

CSV would be a good starting point.

> Right.  But until this is fixed at the top, I see little hope it  
> will happen in the lower levels, except sporadically.

Fixing this problem is likely to become an imperative.  SPF is worse  
than wrong.  It does not offer a safe form of identification, as  
wrongly advertised, and puts DNS and SMTP at risk.  Those advocating  
acceptance based on just the DKIM domain clearly expect to combine  
valid signature with SPF authorization passing.  This is not how the  
DKIM/SPF combination has been being advertised to work together.   
Email is almost certain to become even more unreliable and more  
expensive to operate. :^(

> (The places that do do it are exceptional, and, in the cases where  
> I'm in a position to know why they do it, they do it not because  
> they are held accountable by whoever assigned the resources to them  
> but because they are ethical enough to feel a compulsion to do  
> what's right even when they're _not_ overtly held accountable.   
> While this mindset is common enough for us to have words for it, it  
> is not nearly common enough to save the net from the disasters that  
> governmental disconnect between authority and responsibility leads  
> to.)

One might hope they'll do the right thing out of self preservation.    
Backing schemes that advantage their size only makes problems worse.

-Doug

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 20 Jun 2009 21:20:14 +0200

Douglas Otis wrote:
> SMTP is heavily abused, and soon IPv6 is about to become a necessity.   
> To remain practical, connectivity must be based upon _immediate_ and 
> _stable_ evidence of legitimate email operation, and not upon any number 
> of authorization transactions.  Each additional transaction to support 
> an authorization scheme will be multiplied by the typical number of 
> attempts made by abusive senders.   This means providers need to exclude 
> problematic users, and not become a task pushed toward recipients.  Such 
> pushing is not practical and often leads to unfortunate mistakes.

What do you mean by "problematic users"? Providers of residential 
cables, WiMAX, and similar connections could block or redirect port 
25, just like most universities and companies do. They used to do it, 
as long as they provided mailboxes as a bonus and ISP and ESP were 
synonyms. Submission port 587 is not yet universally employed, and 
some customer may not accept to be unable to reach their favorite 
server's ports 25 or 465. "Blocking port 25 except for a set of 
servers used for submission" is not something that can be easily 
defined and maintained by ISPs, IMHO.

OTOH, sender identification by domain could also be a way to attribute 
responsibility. Strictly speaking, it is not necessary to use a domain 
in order to send as an SMTP client. However, in practice one needs an 
email address to do any legitimate use of SMTP, and hence a domain is 
required.

+1

> provider just to have their email accepted, does not mean other messages 
> emitted might not be mistaken as also belonging to that user's domain.  
> Should providers check for SPF or Sender-ID compliance?  How many SLA 
> include this requirement?  When the "passing-on" is based upon 
> receptions at spam traps, acceptance reliance based on "authorization" 
> is likely to downgrade acceptance of the domain, especially when A-R 
> headers exclude the IP address of the provider.  Will providers really 
> care the wrong entity had been blamed?

You can never know whether that domain's owners are really so foolish 
to trust a criminal provider, rather than participating accessories. 
Assuming their bad faith, one should downgrade acceptance of their domain.

> assurance, especially "moderately strong".

IMHO, the domain registrants (resulting from whois records) provide an 
identification that is comparable in strength, but finer in granularity.

---

From: Franck Martin <[email-masked]>
Date: Sun, 21 Jun 2009 09:33:27 +1200 (FJT)

----- "Alessandro Vesely" <[email-masked]> wrote:

> necessity.   

yes I'm not sure that blocking port 25 will ever be possible. I think less and less people want their mailbox tied up to an ISP, this is why they get a mailbox on yahoo, google, etc... So these services requires you usualy to connect via port 25 and authenticate, but that means for the ISP to let port 25 open. Blocking port 25 and letting port smtps/465 open to allow users to still submit email is better, but just a temporaray measures until botnet use smtps to submit.

The only think I see in this system, is to identify IPs of mail servers via an out of band process. Like a record in the DNS. To avoid DDNS (the ability of the compromised machine to push a record in the DNS), it should be in the Reverse DNS or in a subdomain.

Now a receiving MTA would be able to use this filter, either the sending MTA authenticate (MUA) or the sending MTA is recorded as a MTA in the DNS. Now this cannot be enabled overnight, but a spamassassin filter could give a negative score if the sending MTA is DNS recorded.

---

From: Steve Atkins <[email-masked]>
Date: Sat, 20 Jun 2009 14:47:43 -0700

On Jun 20, 2009, at 2:33 PM, Franck Martin wrote:

> yes I'm not sure that blocking port 25 will ever be possible. I  
> think less and less people want their mailbox tied up to an ISP,  
> this is why they get a mailbox on yahoo, google, etc... So these  
> services requires you usualy to connect via port 25 and authenticate,

Nope, port 587.

> but that means for the ISP to let port 25 open. Blocking port 25 and  
> letting port smtps/465 open to allow users to still submit email is  
> better, but just a temporaray measures until botnet use smtps to  
> submit.

You're conflating two quite different things here, SMTP submission and  
SMTP delivery.

Blocking port 25 outbound (and ideally, inbound) allows an ISP to  
prevent their customers from delivering email directly to recipient  
MXes. It does not prevent their customers from using third-party  
smarthosts at all, as everyone who is intentionally running a third  
party smarthost is listening on port 587.

Bots using port 587 (not 465, that's mostly obsolete) to submit mail  
is a wholly different issue. A bot doing that needs credentials to do  
so (a username and password) and misuse of those credentials will lead  
to them being revoked.

Cheers,
   Steve

---

From: David Nicol <[email-masked]>
Date: Sat, 20 Jun 2009 18:06:08 -0500

On Sat, Jun 20, 2009 at 2:20 PM, Alessandro Vesely<[email-masked]> wrote:
> However, in practice one needs an email address to do any legitimate use of
> SMTP, and hence a domain is required.

There isn't any technical reason why MUA software can't do its own
delivery with today's technology.  The whole concept of "outbound MTA"
seems like a throwback. With the exception of delivery delays, but
having a MUA attempt its own delivery and then switch to the smarthost
on connection or other temporary failure wouldn't require any
inspiration.

As if web-service MUAs are going to fade away, which is doubtful.

-- 
"Of course you don't find it puzzling, YOU WROTE IT!" -- Michael G. Schwern

---

From: Steve Atkins <[email-masked]>
Date: Sat, 20 Jun 2009 16:22:37 -0700

> on connection or other temporary failure wouldn't require any

Just a terminology / architecture comment...

It'd be perfectly possible to build a mail client that behaved that way
(and people have, I think), or setup a system that works in that way
using existing components (and many people do).

But I'd claim that that mail client is then behaving as both an MUA and
an MTA (and probably an MSA too).

It's useful to draw a distinction between the architectural roles, even
when the roles are performed by the same executable.

http://www.bbiw.net/specifications/draft-crocker-email-arch-14.html
is worth a read, ideally with a drink close to hand, for lots of
terminology discussion.

Cheers,
   Steve

---

From: Paul Russell <[email-masked]>
Date: Sat, 20 Jun 2009 22:36:52 -0400

On 6/20/2009 17:33, Franck Martin wrote:
> yes I'm not sure that blocking port 25 will ever be possible. I think less and 
> less people want their mailbox tied up to an ISP, this is why they get a
> mailbox on yahoo, google, etc... So these services requires you usualy to
> connect via port 25 and authenticate, but that means for the ISP to let port
> 25 open. Blocking port 25 and letting port smtps/465 open to allow users to
> still submit email is better, but just a temporaray measures until botnet use
> smtps to submit.

Gmail supports secure authenticated message submission via port 587.

---

From: Alessandro Vesely <[email-masked]>
Date: Sun, 21 Jun 2009 08:11:10 +0200

David Nicol wrote:
> seems like a throwback.

I'd agree that the reasons why we needed outbound MTAs then, and why 
we need'em now are completely different. However, I'd still call the 
current reason "technical"; in facts, its origin is likely related 
with overcoming the former reason.

> With the exception of delivery delays, but

(W.r.t. Steve's terminology / architecture comment, I understand "MUA" 
above as a conceptually personal host, e.g. a laptop running mutt and 
exim.) Being suspicious about their ESPs, those users may prefer to 
deliver directly whenever possible. That is the only reason I see for 
attempting a direct delivery first, and it is a largely suboptimal 
solution anyway (e.g. it provides no reliable storage for sent messages.)

Some people even run MTAs on dynamically assigned IPs. At any rate, 
they use an email address featuring a domain (that they possibly own.) 
That is the domain we should use to work out responsibility.

> As if web-service MUAs are going to fade away, which is doubtful.

They won't go away, but they are fine, since they require proper 
authentication. They relay via submission servers that possibly run on 
the same host or LAN.

---

From: Douglas Otis <[email-masked]>
Date: Sun, 21 Jun 2009 23:34:16 -0700

On Jun 20, 2009, at 12:20 PM, Alessandro Vesely wrote:

>> _immediate_ and _stable_ evidence of legitimate email operation,  
>> and not upon any number of authorization transactions.  Each  
>> additional transaction to support an authorization scheme will be  
>> multiplied by the typical number of attempts made by abusive  
>> senders.   This means providers need to exclude problematic users,  
>> and not become a task pushed toward recipients.  Such pushing is  
>> not practical and often leads to unfortunate mistakes.
>
> 25, just like most universities and companies do. They used to do  
> it, as long as they provided mailboxes as a bonus and ISP and ESP  
> were synonyms. Submission port 587 is not yet universally employed,  
> and some customer may not accept to be unable to reach their  
> favorite server's ports 25 or 465. "Blocking port 25 except for a  
> set of servers used for submission" is not something that can be  
> easily defined and maintained by ISPs, IMHO.

Each recipient will likely attempt to accept either none or some  
amount of public email based upon normal profiles.  To remain on the  
safe size typical profiling, this requires the output issued by bad  
actors be mitigated in some fashion.  This might be done by using rate  
limiting combined with disabling accounts faster than bad actors can  
re-subscribe.  Funny how little anti-spam efforts concentrate on  
account setup.  Things like Open-ID might help in this area, for  
example.

> OTOH, sender identification by domain could also be a way to  
> attribute responsibility. Strictly speaking, it is not necessary to  
> use a domain in order to send as an SMTP client. However, in  
> practice one needs an email address to do any legitimate use of  

Technically speaking, a domain is not required for SMTP.   CSV was to  
offer a DNS record type that explicitly declared a host as being an  
outbound MTA.  This would not in itself prevent abuse, but would help  
to determine which compromised systems might be sending email and  
resolving which domain is administrating the MTA.

SPF does not work well at resolving a domain that should be held  
accountable for a few reasons-

  a) risks high and impractical transaction overheads at attempts to  
indirectly reference the customers of a provider.

  b) may not qualify any specific IP address for a positive result.

  c) Mail From or PRA references do not resolve which domain  
administered the MTA or actually sent the message.

  d) holds customers of a provider accountable for the provider's  
stewardship without any solid evidence of their involvement.

>>>> Schemes that pass accountability onto what might be feckless  
>>>> domain owners are inherently evil.
>>>

There should be greater concern accountability is correctly applied.

>> The fact that a trusting and naive user had their domain authorize  
>> a provider just to have their email accepted, does not mean other  
>> compliance?  How many SLA include this requirement?  When the  
>> "passing-on" is based upon receptions at spam traps, acceptance  
>> reliance based on "authorization" is likely to downgrade acceptance  
>> of the domain, especially when A-R headers exclude the IP address  
>> of the provider.  Will providers really care the wrong entity had  
>> been blamed?
>
> You can never know whether that domain's owners are really so  
> foolish to trust a criminal provider, rather than participating  
> accessories. Assuming their bad faith, one should downgrade  
> acceptance of their domain.

Who said providers need to be criminal for naive users to be harmed by  
SPF?  A recipient may check PRAs, where providers may check Mail- 
Froms.  Once a user's domain reputation is damaged due to receiver  
error, how can reputations be restored and then protected?  When asked  
in the past, those customers are advised to obtain their own IP address.

>> Moderately strong?  Without knowing the IP address of the provider,  
>> it would be extremely foolish to conclude any level of  
>> identification assurance, especially "moderately strong".
>
> IMHO, the domain registrants (resulting from whois records) provide  
> an identification that is comparable in strength, but finer in  
> granularity.

It is wrong to hold someone accountable for authorizing a provider  
once authorization becomes a requisite for acceptance.  Users are  
thereby extorted into assuming risks well beyond their control.

Instead, providers should be held accountable by requiring CSV records  
with a limited number of EHLO host names over time.

This approach better defends receiving MTAs from abuse with lower  
overhead, and better controls DNS related exploits that threaten the  
entire Internet.

-Doug

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 22 Jun 2009 10:57:04 +0100

--On 19 June 2009 11:23:42 -0600 "J.D. Falk" 
<[email-masked]> wrote:

> If the ASRG were to publish some research showing that this is a good
> idea, it'd go a long way towards convincing one of those organizations
> you mentioned to consider implementing it.

So, to demonstrate that it's a good idea, we need to show:

1. That there's lots of spam on the Internet, people don't like it, and it 
costs lots of time and money.

2. The spam is hard to identify, because there's no traceability. People 
like to whitelist or blacklist sender email address or sender email 
domains, but they're too easy to spoof because of the lack of traceability.

3. It would be nice to require traceability for new mailers, but it's hard 
to know which are new.

4. Except that IPv6 mailers are new, or recently deployed.

5. People with recently deployed IPv6 mailers are likely to have the 
ability to implement traceability.

6. The cost to the community of requiring traceability for IPv6 email 
servers would be low.

7. The reward would be that a whole class of easily identified new servers 
would have the traceability required.

8. When deploying IPv6 to customers, ISPs would have to take no special 
measures to prevent customer machines from emitting spam. They'd be 
secure(ish) by default.

Oh, and we have to figure out what form of traceability we're looking for. 
Let's start off with this list for a suggestion:

1. Reverse DNS records for the sender's IP address.
2. SPF or DKIM passes for the sender's IP address.
3. Strict checks on EHLO string.

And, for an IPv6 hosts receiving email, there must be an MX record.

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 22 Jun 2009 11:39:38 +0100

--On 19 June 2009 12:54:17 +0200 Alessandro Vesely <[email-masked]> wrote:

> What about the other way around: given a domain and an IP address, can we
> say whether the IP address "is a member of" the domain?

"is a member of" isn't a useful description of relationships between IP 
addresses and domain names.

You can say that an IP address "is a member of" a netblock. You can say 
that a domain name "is a subdomain" of another domain, and could regard 
that as a "membership" relation.

The DNS is used to express relationships between IP addresses and domain 
names, but there are many types of relationship - like MX records, A 
records. "is a member of" sounds like it might mean "is owned by" or "is 
assigned to", but IP addresses are assigned to real world organisations, 
not domain names. There's no necessary relationship when sending SMTP, 
unfortunately.

You could check whether the IP address and the domain name were assigned to 
the same organisation, but that often won't be possible.

> apparently resorting to heuristics. I'm wondering how sound it is to rely
> on those, or similar, techniques.
>

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 22 Jun 2009 11:41:00 +0100

--On 19 June 2009 11:30:13 -0700 Douglas Otis <[email-masked]> wrote:

> MTP is heavily abused, and soon IPv6 is about to become a necessity.   To
> remain practical, connectivity must be based upon _immediate_ and
> of authorization transactions.  Each additional transaction to support an
> authorization scheme will be multiplied by the typical number of attempts
> made by abusive senders.

Except that results can be cached.

>  This means providers need to exclude

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 22 Jun 2009 11:45:35 +0100

--On 20 June 2009 21:20:14 +0200 Alessandro Vesely <[email-masked]> wrote:

> Submission port 587 is not yet universally employed

No, but widely enough that we've not had any complaints. It's even the 
default port used by some MUAs now - like Apple's "Mail.app" and the 
iPhone. Blocking port 25 will encourage deployment of port 587.

In my view, network operators should assume now that email service 
providers DO provide port 587, unless they have an overwhelming business 
case to do otherwise.

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 22 Jun 2009 12:58:06 +0200

Douglas Otis wrote:
> Each recipient will likely attempt to accept either none or some amount 
> of public email based upon normal profiles.  To remain on the safe size 
> typical profiling, this requires the output issued by bad actors be 
> mitigated in some fashion.  This might be done by using rate limiting 
> combined with disabling accounts faster than bad actors can 
> re-subscribe.  Funny how little anti-spam efforts concentrate on account 
> setup.  Things like Open-ID might help in this area, for example.

Hm... residential accounts with asymmetric rates already complain 
about latency. For example, I've been setting a number of extensions 
to deliver to the Sent folder, so as to allow to save sent mail by Bcc 
rather than copying, for clients that allow it, in order to save time 
in case of large attachments.

As for forcefully disabling accounts, one needs a legal authority to 
break a commercial agreement. For some reasons, they tend to move 
quicker for child porn than spammers. Last time I talked about spam to 
our postal police, the first thing he said has been "me too". Since we 
lack police resources for snatches and rapes, I felt arrogant asking 
them to go after my spammer, and gave up.

I don't follow how OpenId would help.

> CSV was to offer a DNS record type that explicitly declared a host as being 
> an outbound MTA.  This would not in itself prevent abuse, but would help to 
> determine which compromised systems might be sending email and resolving 
> which domain is administrating the MTA.

It didn't help retrieving the responsible domain, though. Furthermore, 
it didn't do ranges, thus making it laborious to deny sending to a 
bunch of hosts. In addition, CSA spec curiously had a "MAY" for an 
non-authorized host with a non-ignored target resulting in weight 0.

However, _client._smtp.domain-name records authorizing various targets 
would be almost equivalent to setting SPF, or MX heuristics. (I'll 
possibly add that to the next vhlo draft.)

> accountable for a few reasons-

One reason is that it works the other way around: given the domain, 
authenticate an IP address as a mail transmitter (thus implicitly 
authorizing it to send mail).

> indirectly reference the customers of a provider.

I agree it is more complicated than the average users need. 
Presumably,  the spec is so for being usable in a wider number of 
cases. 10 lookups, however, seems reasonable to me.

>  b) may not qualify any specific IP address for a positive result.

Yes, especially if not used.

>  c) Mail From or PRA references do not resolve which domain administered 
> the MTA or actually sent the message.

AFAIK, all discussions eventually reach the conclusion that the 
receiving server does not know which domain administers the client 
MTA. FWIW the old ietf-marid-csv-csa asserts that

   The SMTP [RFC2821] [RFC0821] protocol permits a client to declare
   its affiliation, by asserting a domain name in the HELO or EHLO
   announcement.

which is wrong. EHLO only allows it to declare its identity. Declaring 
the affiliation is VHLO's raison d'être.

At any rate, forwarding -when MAIL FROM and PRA differ- requires a 
framework different from SMTP to administer legitimate use of third 
parties' email addresses. Classic mailing lists (as this one) comply 
with various privacy laws, but most direct marketing lists and users' 
.forward files don't.

> stewardship without any solid evidence of their involvement.

See below (at Why do you so often mention authorizing a provider?)

> SPF?  A recipient may check PRAs, where providers may check Mail-Froms.

Formally, SPF only recommends to check MAIL FROM. Any other use of the 
underlying mechanism is just heuristics.

> Once a user's domain reputation is damaged due to receiver error, how 
> can reputations be restored and then protected?

That has nothing to do with SPF. People getting mad about stopping 
spam will invent _any_ rule. I get a score 1.245 for HOST_EQ_IT in 
Fred's Header rules, even if the .it ccTLD never had an SPF record.
(Concentration camps are further down that road.)

>  When asked in the past, 
> those customers are advised to obtain their own IP address.

If responsibility were attributed correctly, this wouldn't be needed 
any more. Of course, if I had an address @spammer, I'd have to share 
their fate, but my address is my choice.

>>> [...]  Without knowing the IP address of the provider, 
>>> it would be extremely foolish to conclude any level of identification 
> It is wrong to hold someone accountable for authorizing a provider once 
> authorization becomes a requisite for acceptance.  Users are thereby 
> extorted into assuming risks well beyond their control.

Why do you so often mention authorizing a provider? ISP and ESP are 
two different activities, although didn't use to. If I use a vanity 
address I have to trust an ESP who provides that service and set MX 
and SPF RRs accordingly. The ESP may or may not be my connection 
provider. The risk is related with choosing the ESP correctly, so that 
trust is well put. That trust is not different for users of an 
existing domain, who don't control the domain, but have to trust its 
admins.

> with a limited number of EHLO host names over time.

But the targets of those records need an address too. What's the 
difference between writing an IP number in an SPF rather than an A RR?

> overhead, and better controls DNS related exploits that threaten the 

All of them rely on DNS, and some posts in that "DNS over SCTP" thread 
showed that various people think just of http when they figure out 
"typical" DNS usage... :-(

---

From: der Mouse <[email-masked]>
Date: Mon, 22 Jun 2009 07:20:46 -0400 (EDT)

> break a commercial agreement.

Maybe that's why ISP/client contracts - at least in my admittedly
limited experience - permit the ISP to terminate service for any reason
whatever and be liable for, at most, the last billing period's charges:
to permit them the latitude to cut off abusers without having to go
through the legal system first.  (Of course, taking advantage of that
frivolously is an excellent way to lose their customer base, and I'm
sure they know it.)

> For some reasons, they tend to move quicker for child porn than
> spammers.  Last time I talked about spam to our postal police, the
> first thing he said has been "me too".  Since we lack police
> resources for snatches and rapes, I felt arrogant asking them to go
> after my spammer, and gave up.

If your legal system does not provide for private right of action
against spammers, it needs fixing, and this is exactly why.

It also points up part of why depending on meatspace law to deal with
net offenses is a bad idea.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 22 Jun 2009 14:35:56 +0200

Ian Eiloart wrote:
> [...]
> records.

A records. MX bear no IP address. Other records may hold an IP 
address, e.g. TXT, thus providing possibly weaker relationships.

> "is a member of" sounds like it might mean "is owned by" or "is 
> not domain names.

You're right, the admins of a domain may put whatever A records in 
their zone files. I have to add that I get the domain name _from_ the 
given IP. In that case, if I'm able to find a record in the domain's 
zone that confirms that relationship, can I safely deduce that the 
membership relation holds?

> There's no necessary relationship when sending SMTP, unfortunately.

Agreed. But why do you say "unfortunately"? Do you mean that it would 
always be preferable to attribute responsibility based on the IP 
delegation hierarchy, rather than on the names' one, or have we always 
tried to go the former way just because the IP address of the remote 
host is easier to obtain?

---

From: John Levine <[email-masked]>
Date: 22 Jun 2009 12:52:41 -0000

>we say whether the IP address "is a member of" the domain?

In a word, no.  SPF is the best effort we have along those lines, and
for reasons I hope I don't have to reiterate, it's far from adequate.

R's,
John

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 22 Jun 2009 14:59:01 +0100

--On 22 June 2009 14:35:56 +0200 Alessandro Vesely <[email-masked]> wrote:

> A records. MX bear no IP address. Other records may hold an IP address,
> e.g. TXT, thus providing possibly weaker relationships.
>
> You're right, the admins of a domain may put whatever A records in their
> zone files. I have to add that I get the domain name _from_ the given IP.
> In that case, if I'm able to find a record in the domain's zone that
> confirms that relationship, can I safely deduce that the membership
> relation holds?
>
> tried to go the former way just because the IP address of the remote host
> is easier to obtain?

We use IP address reputation services because there's nothing else we can 
use, in the absence of some way to authenticate the sender address. Of 
course, those mechanisms exist and are widely deployed but not universally, 
or even by a majority of domains. When they become so, we'll no doubt see 
domain based reputation services, and even address based reputation 
services being used as much as IP address reputation services are.

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 22 Jun 2009 15:12:08 +0100

> On Jun 20, 2009, at 12:20 PM, Alessandro Vesely wrote:
...
> outbound MTA.  This would not in itself prevent abuse, but would help to

Er, we already have ridiculous transaction overheads for email. Anything 
that stopped spam would reduce the transaction overheads for legitimate 
email by up to ten fold.

>   b) may not qualify any specific IP address for a positive result.

I'm not sure what that phrase means. If it means that some lookups result 
in softfail or neutral results, then that actually doesn't matter much. The 
passes and the fails still get us useful information. Anything else just 
puts us back where we were before.

> the MTA or actually sent the message.

It doesn't matter. If the domain owner devolves responsibility to the IP 
address owner, then the mail is effectively from the domain owner, and they 
can be held responsible for their email. Reputation services, and the law 
can be applied as appropriate.

> stewardship without any solid evidence of their involvement.

Please expand, I don't understand this either.

> There should be greater concern accountability is correctly applied.

If the domain owners are feckless, then apply sanctions. Accountability HAS 
to lie with domain owners if you want to establish reputation services 
based on domain names, and most people do want to do that. If the domain 
owner is found to be feckless, then reputation sanctions should be applied.

---

From: John Leslie <[email-masked]>
Date: Mon, 22 Jun 2009 13:52:31 -0400

Alessandro Vesely <[email-masked]> wrote:
> Douglas Otis wrote:
> 
>> CSV was to offer a DNS record type that explicitly declared a host as 
>> being an outbound MTA.  This would not in itself prevent abuse, but would 
>> help to determine which compromised systems might be sending email and 
> It didn't help retrieving the responsible domain, though.

   CSV intended to use the EHLO string _as_ the responsible domain, even
though it might be one of several subdomains under a single management.

> Furthermore, it didn't do ranges, thus making it laborious to deny
> sending to a bunch of hosts.

   I agree it didn't "do ranges", since it seemed appropriate to allow
separate reputations for separate EHLO names.

> In addition, CSA spec curiously had a "MAY" for an non-authorized host
> with a non-ignored target resulting in weight 0.

   That was an artifact of our being told that excessive queries for
the root "." would cause problems. The "MAY" covered how to program a
case which "should" never happen.

> possibly add that to the next vhlo draft.)

   I'm certainly game to do any needed updates to draft...csv...

> MTA.

   CSV was intended to address exactly that problem. We felt that the
vast majority of domains _could_ get by with half a dozen IP addresses
returned by a A)ddress record, and that for those that couldn't, the
ability to have separable reputations was an _advantage_. For a
reputation service, more than one reputation record per domain actively
sending (reasonable) email seemed simple enough.

> FWIW the old ietf-marid-csv-csa asserts that
> 
> which is wrong. EHLO only allows it to declare its identity.

   It's a relief, actually, to _finally_ have a useful correction to
these drafts!

   I agree that "affiliation" was not the best choice of words in
draft-ietf-marid-csv-csa-02. "Affiliation" should be multi-valued,
with an individual sending MTA able to affiliate with multiple
entities.

   OTOH, "identity" isn't quite right either. Though it is the term
used in RFC 2821, that RFC in no sense requires that the "identity"
identify a single server.

   "Identity" would probably be better that "affiliation"; some other
term might be better than either...

> Declaring the affiliation is VHLO's raison d'?tre.

   Hmm... I still think "affiliation" deserves to be multi-valued...

--
John Leslie <[email-masked]>

---

From: Douglas Otis <[email-masked]>
Date: Mon, 22 Jun 2009 11:16:18 -0700

On Jun 22, 2009, at 3:58 AM, Alessandro Vesely wrote:

> quicker for child porn than spammers. Last time I talked about spam  
> to our postal police, the first thing he said has been "me too".  
> Since we lack police resources for snatches and rapes, I felt  
> arrogant asking them to go after my spammer, and gave up.

This suggests poor Acceptable Use Policies.

> I don't follow how OpenId would help.

Stable identifiers might mitigate some vetting.

>> CSV was to offer a DNS record type that explicitly declared a host  
>> as being an outbound MTA.  This would not in itself prevent abuse,  
>> but would help to determine which compromised systems might be  
>> sending email and resolving which domain is administrating the MTA.
>
> sending to a bunch of hosts. In addition, CSA spec curiously had a  
> "MAY" for an non-authorized host with a non-ignored target resulting  
> in weight 0.

The domain administrating MTAs ARE the truly responsible Domains.  It  
would be speculation whether an entity offering authorization actually  
originated the messages.   CSV was intended to affirm the SMTP  
operation by a specific domain.   Any CSA record not expressing an  
affirmation MAY be assumed an indication of the host not being  
authorized.

> However, _client._smtp.domain-name records authorizing various  
> targets would be almost equivalent to setting SPF, or MX heuristics.  
> (I'll possibly add that to the next vhlo draft.)

Only the HELO mechanism of SPF comes close, but SPF itself does not  
offer a means to limit the SPF record scope.

> authorizing it to send mail).

Authorization does not indicate either administration or origination.   
Any need for authorization records for acceptance will be extorting  
the publication of records that might allow blame to be incorrectly  
placed.   Hardly a way to ensure providers remain accountable. :^(

> cases. 10 lookups, however, seems reasonable to me.

It is actually eleven TXT SPF records that might be needed.  The SPF  
spec allows 10 mechanisms, where there might be a need for 10  
subsequent target resolutions per mechanism .  The actual limit of DNS  
transactions is at 111.

> Yes, especially if not used.

Even when used.  The record can end in +all or use some exists().  SPF  
also allows the use of macros.  These macros can be cached, and yet  
cause recipients to generate different transactions based upon  
changing local-parts or IP addresses, for example.

> its affiliation, by asserting a domain name in the HELO or EHLO  

SMTP (RFC 5321) section 2.3.5 does not stipulate the resolution of the  
domain asserted by EHLO as using A,  AAAA or MX records.  It also  
allows use of address literals.  Section 4.1.4 then indicates  
resolution failure of the EHLO domains or mismatch of address literals  
MUST NOT be cause for refusing messages.  Likewise, SPF also fails to  
impose negative assertions for ELHO domain failures for Mail-From or  
PRA domains, of course.

> Declaring the affiliation is VHLO's raison d'être.

Since VHLO can be asserted by bad actors, there is no reason to trust  
the guidance offered.

> with various privacy laws, but most direct marketing lists and  
> users' .forward files don't.

Trusting the MTA via CSV/CSA does not impact message handling, unlike  
problems created by Sender-ID and SPF.

> See below (at Why do you so often mention authorizing a provider?)

Most domains outsource email services.  This is a growing trend as  
email becomes more heavily abused.

>> Who said providers need to be criminal for naive users to be harmed  
>> by SPF?  A recipient may check PRAs, where providers may check Mail- 
>> Froms.
>
> Formally, SPF only recommends to check MAIL FROM. Any other use of  
> the underlying mechanism is just heuristics.

The lackadaisical attitude ends abruptly when reputation is misapplied  
as a result of unverifiable assumptions of message origination when  
erroneously based upon authorizations.  This would be extending SPF  
well beyond its intended purpose.

>> Once a user's domain reputation is damaged due to receiver error,  
>> how can reputations be restored and then protected?
>
> That has nothing to do with SPF.

Any negative assertions based upon SPF might be misapplied against  
PRAs not protected by the provider.  Misapplication of negative  
assertions might also be due to SPF when Mail-Froms are also not  
protected by the provider.  Have you ever seen a SLA that stipulates  
PRA or Mail-From exclusivity?

>> When asked in the past, those customers are advised to obtain their  
>> own IP address.
>

If the IP address were only associated with that of the MTAs, users  
would be able to seek services from any other provider.  When a  
provider's lack of stewardship impacts user's domains, then users are  
left without resource, while the provider remains free to seek more  
victims.

> Why do you so often mention authorizing a provider?

SPF is a scheme aimed at authorizing providers and thereby shifting  
blame.

> If I use a vanity address I have to trust an ESP who provides that  
> service and set MX and SPF RRs accordingly.

Why do you feel there is a need to publish SPF records?

> The ESP may or may not be my connection provider. The risk is  
> related with choosing the ESP correctly, so that trust is well put.  
> That trust is not different for users of an existing domain, who  
> don't control the domain, but have to trust its admins.

Of course, trusting an ESP to keep incoming email confidential is  
important, and is often covered in SLAs and various laws.   But why  
are users required to blindly trust that email providers will filter  
other users outbound email?  It is unlikely that this filtering is  
covered by SLAs or by various laws.

>> Instead, providers should be held accountable by requiring CSV  
>> records with a limited number of EHLO host names over time.
>

CSV is only about who is administrating the MTA.  It has nothing to do  
with who might be inadvertently held responsible based upon Mail-Froms  
or the PRAs.

>> overhead, and better controls DNS related exploits that threaten  
>> the entire Internet.
>
> All of them rely on DNS, and some posts in that "DNS over SCTP"  
> thread showed that various people think just of http when they  
> figure out "typical" DNS usage... :-(

Unfortunately, when risks associated with SPF with respect to DNS was  
explained, SPF advocates expressed distain for DNS.

-Doug

---

From: Douglas Otis <[email-masked]>
Date: Mon, 22 Jun 2009 12:05:00 -0700

On Jun 22, 2009, at 7:12 AM, Ian Eiloart wrote:

> Er, we already have ridiculous transaction overheads for email.  
> Anything that stopped spam would reduce the transaction overheads  
> for legitimate email by up to ten fold.

Only the application of reputation and address range policies reduces  
spam levels.  Not using SPF and instead using CSV will reduce the  
transaction overhead needed to validate an associated domain.  SPF  
often requires several transactions, that may exceed several hundred  
transactions where 111 could be generated by PRAs and then another 111  
for the Mail-From.   The high overhead problem of SPF can be made  
worse when the SPF records contain macros.  Using SPF macros, bad  
actors can cause recipients to generate a long series of different DNS  
transactions based upon portions of an email-address local-part, for  
example.  This enables a free DDoS attack while spamming, since SPF  
macros can make DNS caching ineffective.

> I'm not sure what that phrase means. If it means that some lookups  
> result in softfail or neutral results, then that actually doesn't  
> matter much. The passes and the fails still get us useful  
> information. Anything else just puts us back where we were before.

An SPF pass result may never be based upon the IP address of the MTA.   
The way SPF is defined, there might not be a means to know whether an  
IP address check is bogus when resolving exists functions, for example.

> It doesn't matter. If the domain owner devolves responsibility to  
> the IP address owner, then the mail is effectively from the domain  
> owner, and they can be held responsible for their email. Reputation  
> services, and the law can be applied as appropriate.

You have this the wrong way around.  SPF devolves responsibility to  
that of the email-address domain owner from that of the provider.  Any  
conclusions about email origination based upon authorization is purely  
speculative.  It would be wrong,  perhaps even risky, to conclude a  
message originated from a domain offering MTA authorization.

> Please expand, I don't understand this either.

Providers are not required to assure any particular email-address  
domain within the Mail-From or that of some PRA is used exclusively by  
users of the email-address domain.  There may not be a relationship  
between the account used to gain access to an email providers'  
outbound MTA and the domain of SPF record authorizing the MTA.

> If the domain owners are feckless, then apply sanctions.  
> Accountability HAS to lie with domain owners if you want to  
> establish reputation services based on domain names, and most people  
> do want to do that. If the domain owner is found to be feckless,  
> then reputation sanctions should be applied.

CSV better ensures the domain providing access is held accountable.   
Accountability is normally applied against the IP address of the MTA.   
SPF's attempt at holding Mail From or the PRA domains accountable  
subverts the often intended use of SPF as a means to mitigate  
backscatter.  It would be foolish to assume SPF assigns who is  
accountable for having originated a message.

-Doug

---

From: Dotzero <[email-masked]>
Date: Mon, 22 Jun 2009 16:13:37 -0400

On Mon, Jun 22, 2009 at 3:05 PM, Douglas Otis<[email-masked]> wrote:
> Only the application of reputation and address range policies reduces spam
> levels.  Not using SPF and instead using CSV will reduce the transaction
> overhead needed to validate an associated domain.  SPF often requires
> several transactions, that may exceed several hundred transactions where 111
> could be generated by PRAs and then another 111 for the Mail-From.   The
> high overhead problem of SPF can be made worse when the SPF records contain
> macros.  Using SPF macros, bad actors can cause recipients to generate a
> long series of different DNS transactions based upon portions of an
> email-address local-part, for example.  This enables a free DDoS attack
> while spamming, since SPF macros can make DNS caching ineffective.
>

Doug,

I'd take your discussions of SPF more seriously if you would stop
conflating SPF and Sender-ID. They are two different animals. SPF (the
specification) does not include anything called PRA. Sender-ID
includes the concept of PRA. PRA is broken in the spec so there isn't
any purpose in spending time discussing it. All one needs to do is
look at the paragraph that states that if a sender field exists you
set the PRA to that. This bypasses any SPF record published for the
Mail From (envelope sender) domain. End of discussion.

---

From: Rich Kulawiec <[email-masked]>
Date: Mon, 22 Jun 2009 17:53:54 -0400

On Mon, Jun 22, 2009 at 02:59:01PM +0100, Ian Eiloart wrote:
> course, those mechanisms exist and are widely deployed but not 
> universally, or even by a majority of domains. When they become so, we'll 
> no doubt see domain based reputation services, and even address based 
> reputation services being used as much as IP address reputation services 
> are.

I don't think so.  Domains and addresses are nearly-free and disposable,
so spammers could easily render both pointless exercises whenever it
suited them to do so.  Given that registrars are quite happy to continue
selling dirt-cheap domains by the thousands to even the worst spammers
(and registrars ARE spammers) it will always be possible for abusers to
come up with another domain and another email address -- or another ten
thousand of each -- whenever it suits them.   Network space is not quite
so easy to come by, so I think we stand a better chance keeping track of
allocations.

---Rsk

---

From: Steve Atkins <[email-masked]>
Date: Mon, 22 Jun 2009 15:08:17 -0700

> so easy to come by, so I think we stand a better chance keeping  

The critical point here is that while it's easy to cycle through  
domains,
only those who are doing Bad Stuff will do so.

If you're sending wanted email then the reputation associated with any
reputation key (including domains) will increase, and quality of  
delivery
will continue to improve.

If you're sending unwanted email then the associated reputation will
decrease and delivery rates will drop. Because of that, people sending
bad email will cycle through reputation identifiers rapidly, meaning  
that
their reputation is never better than that of a brand new identifier,  
but not
usually much worse.

That makes reputation of this sort (whether it be IP based,  
authenticated
domain based or anything else where it's easy to create a new reputation
key, but hard to steal someone elses) is extremely useful for  
identifying
mail that's likely to be wanted, and not really great for identifying  
mail that's
likely to be unwanted. It's not something that's useful on it's own,  
but it's
incredibly useful when used in conjunction with other approaches.

Cheers,
   Steve

---

From: Douglas Otis <[email-masked]>
Date: Mon, 22 Jun 2009 15:45:31 -0700

On Jun 22, 2009, at 1:13 PM, Dotzero wrote:

> conflating SPF and Sender-ID. They are two different animals. SPF  
> (the specification) does not include anything called PRA. Sender-ID  
> includes the concept of PRA. PRA is broken in the spec so there  
> isn't any purpose in spending time discussing it. All one needs to  
> do is look at the paragraph that states that if a sender field  
> exists you set the PRA to that. This bypasses any SPF record  
> published for the Mail From (envelope sender) domain. End of  
> discussion.

The SPF (RFC 4408) lacks a means to constrain the scope of  
authorization records to just Mail-From.  Sender-ID (RFC 4406) section  
3 allows SPF (RFC 4408) records to authorize MTAs based upon PRA  
headers, and perhaps failing that, upon the Mail-From parameter.  RFC  
4406 section 3.1 modifies RFC 4408 versioning and adds a scope  
parameter.  Per RFC 4406 section 3.4, the unmodified SPF record is to  
be considered the same as spf2.0/mfrom,pra. :^(

Section 3.4 of RFC 4406 also states:
,---
If the information in a "v=spf1" record is not correct for a PRA  
check, administrators SHOULD publish either an "spf2.0/pra" record  
with correct information or an "spf2.0/pra ?all" record indicating  
that the result of a PRA check is explicitly inconclusive.
'---

Per RFC 4406, when PRA authorization fails, an attempt to verify Mail- 
 From authorization might be attempted.  When only RFC 4408 has been  
relied upon, the Mail-From domain may inadvertently offer  
authorization based upon PRA headers.  In this case, how SPF records  
are used is uncontrolled.  From an overhead standpoint, RFC 4406  
potentially doubles the overhead related to SPF record resolution.   
Although RFC 4406 recommends against use of MTAs that "lie" about  
local-parts, it failed to exclude macro use, or to consider what  
happens when providers do not restrict the use of domains in  
originating headers, in addition to the use of local-parts.

Email would have likely faired much better by recommending immediate  
rejection, use of RFC 3464, and minimal DSN content with"multipart/ 
report" content types.

-Doug

---

From: Seth <[email-masked]>
Date: Mon, 22 Jun 2009 23:56:51 -0400 (EDT)

> break a commercial agreement.

You mean the way the spammers get a court order before they violate
their agreement not to spam?

The typical ISP agreement says the ISP can close the account for
abuse, and the ISP gets to determine what is abuse.

Seth

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 23 Jun 2009 08:02:10 +0200

John Leslie wrote:
>> [CSV] didn't help retrieving the responsible domain, though.
> 

Large organizations are often partitioned into (proper) subdomains, 
more or less independent from one another. OTOH single mailout hosts 
names may also be considered a (degenerate) subdomain.

> separate reputations for separate EHLO names.

Interesting. However, it is useless to separate reputation of 
different EHLO names belonging to an array of mailout hosts where a 
specific one is chosen based on traffic optimization and may even 
change on retries.

> sending (reasonable) email seemed simple enough.

OTOH, it is more difficult to track subdomains. A spammer can easily 
add new names at incredible rates. What about DKIM's distinction 
between domain (d) and identity (i)? As clever as DKIM may bee, I 
never noticed mail messages altered in transit, and would be curious 
to know how many filter look at its signature headers for the sole 
purpose of extracting the affiliation.

> identify a single server.

John K. has been very clear on that point. Section 2.3.5 says

  The domain name given in the EHLO command MUST be either a primary
  host name (a domain name that resolves to an address RR) or, if
  the host has no name, an address literal, as described in
  Section 4.1.3 and discussed further in the EHLO discussion of
  Section 4.1.4.

It is true that section 4.1.4 specifies that giving the affiliation 
instead of the host name cannot be punished by rejecting messages. 
However, that wording obviously implies that no reliable affiliation 
information can be derived from the EHLO name. John himself 
suggested to use VHLO if affiliation is required. In his words

    Although I'd predict that you'd have trouble
  getting it off the ground, another possibility would be to
  propose to replace EHLO with a validated-hello command, VHLO,
  which would be just like EHLO except for support for a different
  argument architecture.
[http://www.imc.org/ietf-smtp/mail-archive/msg05444.html]

>> Declaring the affiliation is VHLO's raison d'etre.
> 

In what sense?

In the vhlo draft[1] I define two domains compatible if they share 
at least one primary mail exchanger. Would that be a 
multi-affiliation example?
[1] http://tools.ietf.org/html/draft-vesely-vhlo

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 23 Jun 2009 15:02:22 +0100

--On 22 June 2009 17:53:54 -0400 Rich Kulawiec <[email-masked]> wrote:

>> We use IP address reputation services because there's nothing else we
>> can  use, in the absence of some way to authenticate the sender address.
>> Of   course, those mechanisms exist and are widely deployed but not
>> universally, or even by a majority of domains. When they become so,
>> we'll  no doubt see domain based reputation services, and even address
>> based  reputation services being used as much as IP address reputation
>> services  are.
>
> suited them to do so.

Yes, they are. But, acquiring reputation for a domain is a different 
question. Sure, a new email address doesn't have a negative reputation, but 
it doesn't have a positive reputation either.

Mail server configurations of the future will likely reject email from 
addresses with negative reputations (except where whitelisted), accept 
email from addresses with good positive reputations (except where 
blacklisted), and do other stuff with addresses without reputation 
(including newly registered addresses, and previously unused addresses.

What will they do with the addresses without reputation scores? Well, at 
worst only what they do now - examine the content, check the IP address 
reputation, etc. But, they'll also have a host of other things they can do, 
including domain based whitelisting and blacklisting (pointless without 
authentication). And, they'll be able to - for example - rate limit mail 
from unusual addresses until the new addresses have acquired sufficient 
reputation.

And, if they do use new domains for spam, we can track them through the 
registrars. Unresponsive registrars will acquire poor reputation - so 
expect to see registrar based reputation services, too.

> Given that registrars are quite happy to continue

Yes, but what's the point? I've never had any of my users ask me to 
whitelist an IP address. I've had plenty ask me to whitelist domains and 
specific addresses. We don't do that at the moment, because a whitelist 
entry is simply a hole in our spam defences. Oh, and notice that it hasn't 
actually worked very well.

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 23 Jun 2009 19:39:01 +0200

Steve Atkins wrote:
>>> can use [...]

Maintaining reputation records based on domain is much easier than 
doing single email addresses, especially if one knows what egress 
anti-spam practices they deploy, and what's their policy for creating 
new accounts. In this case, mandating SUBMIT a la SPF shouldn't affect 
a domain's reputation; however, only messages that are relayed that 
way can be whitelisted on the basis that they come from a whitelisted 
domain.

> The critical point here is that while it's easy to cycle through domains,
> reputation key (including domains) will increase, and quality of delivery 

A domain changing ISP or location will most likely get new IP 
addresses. This noise is absent when tracking reputation by domain name.

> bad email will cycle through reputation identifiers rapidly, meaning that
> but not usually much worse.

If whitelists by domain were the rule, newcomers would seek the 
endorsement of their business associations, reputable friends, and 
possibly even employees. They will introduce themselves, and avoid 
whois privacy concealments. Investing in such sort of vernissage for 
new IP addresses makes little sense.

> That makes reputation of this sort (whether it be IP based, authenticated 
> key, but hard to steal someone elses) is extremely useful for identifying 
> mail that's likely to be unwanted. It's not something that's useful on 
> it's own, but it's incredibly useful when used in conjunction with other 
> approaches.

It's only natural to think that mail that's likely to be wanted shall 
take priority, as it does not require Bayesian content filtering, 
waiting for hashes from honeypots, and similar mumbo jumbos. Then, if 
whitelisted domains become widespread, we can peacefully harden the 
rules for filtering the rest.

The point is, if it is easier and convenient, why isn't it plenty of 
RHSWLs out there?

---

From: mathew <[email-masked]>
Date: Thu, 25 Jun 2009 19:26:29 -0500

> I think that as soon as you start quoting the dictionary, you've lost

Exactly what Humpty Dumpty would say. Glory!

mathew

---

From: Bill Cole <[email-masked]>
Date: Mon, 29 Jun 2009 12:35:52 -0400

[...]

> as belonging to them, even when it could improve email acceptance.

Not really. As far as I can tell, provider attention was not controlled for 
in that experiment.

In my experience the awareness of existing Internet standards, the Internet 
standards process, and the "open unstructured testing in production" model 
that dominates the development of new protocol features among people running 
mail systems and companies selling mail services is very low. I would be 
shocked if most providers of mail services even today are aware of the CSV 
proposals' existence at any level, either corporately or among their 
technical staff.

> This

Obviously that plays strongly into the awareness issue.

---

From: Bill Cole <[email-masked]>
Date: Mon, 29 Jun 2009 14:49:41 -0400

> on connection or other temporary failure wouldn't require any

Injecting a concrete fact into the scholastic debate...

The original Eudora (before it became an alpha-quality extension for a beta 
version of Thunderbird) had such functionality going back at least to the 
early 90's. That was not an expression of any sort of advanced modern 
technology, but rather a necessity for the early days of putting a MUA on a 
personal system instead of on a shared system with its own local submission 
and queuing system.

The reasons that younger MUA's do not offer that are social, not technical. 
It has never been particularly difficult technically for a MUA to work 
without a smarthost/MSA, but for the past decade it has been increasingly 
difficult because receiving systems have become less trusting of random 
strangers as clients. It is empirically useful for a receiving system to 
assume that mail offered via SMTP from an IP that is being used by a 
personal computer without useful sender authentication (which might be "this 
IP is on my network") is some variety of spam.

The "outbound MTA" (or more properly: MSA) is not a throwback, but a 
relative novelty. Hypothesizing its elimination without a model for 
eliminating the preconditions that led to it being a standard part of how 
people use mail is pointless. Those preconditions are:

1. There is no working global mechanism for identifying an accountable party 
(i.e. one who explicitly *accepts* accountability) from an IP address, due 
largely to the political and historical variations in how IP addresses have 
been allocated.

2. A large fraction of users who have theoretical control over the traffic 
coming from a public IP address do not have the requisite understanding of 
their own systems and how they use those systems to be accountable for the 
traffic generated by them.

Funneling email through MSA systems run by providers that in principle have 
some means of holding their users accountable and are capable of at least 
understanding bad behavior in mail if not always keeping it controlled is 
the best partial workaround we have, and it implies the need for 
domain-level accountability or its equivalent. The fact that other models 
could work technically is not relevant unless they come with strong 
mechanisms to work around those social gaps.

---

From: John Levine <[email-masked]>
Date: 29 Jun 2009 21:10:16 -0000

>without a smarthost/MSA, ...

Well, for fairly loose definitions of "work", since I've never seen an MUA
that did retries on soft failure.

Now the question is essentially moot, since any legitimate mail sent direct
from an MUA is swamped in the flood from bots.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 30 Jun 2009 09:42:21 +0200

Bill Cole wrote:
> 1. There is no working global mechanism for identifying an accountable 
> party (i.e. one who explicitly *accepts* accountability) from an IP 
> address, due largely to the political and historical variations in how 
> IP addresses have been allocated.

At a first glance, this may seem a flaw in the rDNS/whois systems. 
Upon reconsideration, I realize I have no means to accept 
accountability for an IP address of mines, since SPF or CSV/CSA only 
convey authorization for using a name. In facts, we don't even have 
a term for "the accountable party related to an IP address".

Dave's Email Arch mentions an Originator as "accountable for the 
message content", but doesn't relate it to an IP address. Rfc5068 
associates accountability after submission with traceability 
features of the MSA, apparently suggesting that the first relaying 
thereafter is from an IP which is (indirectly) accountable for the 
message content. Reasoning by induction on the hops, one may 
conclude that all relays using a smarthost are accountable: 
smarthosts require either IP/firewall configuration or 
authentication (assuming they are not open relays.) Accountability 
breaks at the MX-driven relay, often referred as "boundary".

> Funneling email through MSA systems run by providers that in principle 
> have some means of holding their users accountable and are capable of at 
> least understanding bad behavior in mail if not always keeping it 
> controlled is the best partial workaround we have, and it implies the 
> need for domain-level accountability or its equivalent.

Why is it partial?

"Domain-level accountability" is a good approximation. However, a 
smarthost is not necessarily within the same domain (e.g. ukisp.com 
is not even in the same 1st level domain) or the same organization. 
How does accountability degrade through indirection? That is, would 
you trust an SMTP client the same if it relays on behalf of some 
other party?

---

From: John Leslie <[email-masked]>
Date: Tue, 30 Jun 2009 14:40:18 -0400

Bill Cole <[email-masked]> wrote:
> that dominates the development of new protocol features among people 
> running mail systems and companies selling mail services is very low. I 
> would be shocked if most providers of mail services even today are aware of 
> the CSV proposals' existence at any level, either corporately or among 
> their technical staff.

   Available evidence supports that... I've been running CSV on input for
years. Of 5 million recent email sessions on one server, I found 148 CSA
entries, of which only 6 from cheetahmail.com indicated CSV authorization
for the IP address connecting to us.

   For research purposes I list the "not-authorized" HELO domains:

123mail.org
2-mail.com
airpost.net
bestmail.us
cluemail.com
collaborations.com
elitemail.org
eml.cc
f-m.fm
fast-email.com
fastem.com
fastmail.cn
fastmail.co.uk
fastmail.fm
fastmail.net
fastmail.to
fastmail.us
fmgirl.com
imap.cc
letterboxes.org
mail
mailbolt.com
mailc.net
mailforce.net
mailftp.com
mailite.com
mailnew.com
mailservice.ms

---

From: John Leslie <[email-masked]>
Date: Tue, 30 Jun 2009 16:01:50 -0400

Alessandro Vesely <[email-masked]> wrote:
> convey authorization for using a name.

   I'm not certain whether that is a helpful statement.

   SPF doesn't really "convey authorization for using" a name. It sets
out to convey information about which IP addresses are "expected" to
be used by MTAs sending emails "using" a name. But even that evades us
as SPF calls for executing a particular algorithm which returns
pass/fail (or something else...).

   CSV _does_ set out to convey _exactly_ "authorization for using a
name" in the HELO command; but the standard CSA query generally returns
a list of IP addresses which are "authorized" to use that HELO name.

> In facts, we don't even have a term for "the accountable party related
> to an IP address".

   Are you sure that's a useful concept?

   The CSV paradigm is that the operator of a MTA should exercise some
responsibility for what is sends. The HELO string identifies the MTA
(though not necessarily one string exclusively by one MTA), and the
DNS management for that domain-name string states whether that domain
exercises responsibility (and by automatic return of A)ddress RRs on
SRV queries, what IP address(es) that MTA uses).

   While this perhaps comes "close", it's not designating an "accountable
party"; and the IP address is related to the HELO string, not the other
way around. It does _not_ lead to an "accountable party" -- it merely
associates a reference string (the domain name) that we can use as a
query to reputation services.

> message content", but doesn't relate it to an IP address.

   ... or anything else...

> Rfc5068
" 
" This note recommends conventions for the operation of email submission
" and transport services between independent operators, such as
" enterprises and Internet Service Providers.

> associates accountability after submission with traceability features
> of the MSA, apparently suggesting that the first relaying thereafter
> is from an IP which is (indirectly) accountable for the message content.

   Actually,
" 
" Relaying and delivering employ policies that occur after submission and
" are outside the scope of this document.

RFC5068 deals with the operation of Mail Submission Agents. I don't agree
it even "suggests" how accountablity should follow the message as it
winds its way to the recipient.