Subject: Sufficient email authentication requirements for IPv6

From: Douglas Otis <[email-masked]>
Date: Thu, 28 Mar 2013 11:13:42 -0700

Dear IETF,

In response to various strategies to reject IPv6 email lacking either DKIM
or SPF, the non-negotiated approach suggests far greater review is needed.

Here is a paper illustrating problems with DKIM.
https://www.dropbox.com/sh/jh4z407q45qc8dd/MlcUTUFUf4/Domains%20as%20a%20basis%20for%20managing%20traffic.pdf

Rather than offering a means to negotiate, returning a 554 response is seen
as a way to coerce senders to try other MX records.  In
https://tools.ietf.org/html/rfc5321 this code does not clarify why a
connection has been rejected, but implies in the case of a
connection-opening response, "No SMTP service here".

An alternative might be to use existing negotiation techniques for scalable
source authentication:

http://tools.ietf.org/html/rfc4954 offers 530 5.7.0 Authentication required
   This response SHOULD be returned by any command other than AUTH,
   EHLO, HELO, NOOP, RSET, or QUIT when server policy requires
   authentication in order to perform the requested action and
   authentication is not currently in force.
530 seems like a better response.

421 is far more likely to be understood as fallback for problematic
clients, but remembering anything about prior IPv6 clients is unworkable.

http://tools.ietf.org/html/rfc4954 offers a means to properly negotiate
enhanced requirements.  Since DKIM in its current form can not enhance
authentication to a level able to mitigate abuse, it does not justify
negotiation.  SPF is not about authentication.  SPF is an authorization
scheme.

Smarthost services for naive senders new to IPv6 could permit an easy
introduction to scalable authentication schemes like StartTLS.  Formalized
negotiations can solve an abuse problem by placing added burdens on senders
for a scalable scheme that should prove far more robust.  I can expand on
this if anyone is interested.

Regards,
Douglas Otis

---

From: Hector Santos <[email-masked]>
Date: Thu, 28 Mar 2013 18:53:20 -0400

Hi Doug,

> or SPF, the non-negotiated approach suggests far greater review is needed.

Whats the difference with IPv6 connections?  Should it matter? Does it 
matter?

> https://www.dropbox.com/sh/jh4z407q45qc8dd/MlcUTUFUf4/Domains%20as%20a%20basis%20for%20managing%20traffic.pdf

This requires a sign up to obtain/view. Sorry.

> as a way to coerce senders to try other MX records.

Don't follow. A 55x is a permanent rejection. Not a SMTP protocol 
instruction to retry.

> In

It may be, but it may force a client to continue a AUTH sequence and 
thats not possible if ESMTP is not in place or AUTH is not part of the 
ESMTP options.

Sounds like there are apples and oranges being mixed up here.

> clients, but remembering anything about prior IPv6 clients is unworkable.

Don't follow.  SMTP clients are following a SMTP state machine:

    4xx means retry
    5xx means don't retry

It really doesn't matter why. But if you are talking about a 
"greylisting" behavior, in my technical opinion, a SMTP Extension 
proposal such as SMTPGREY:

     http://tools.ietf.org/html/draft-santos-smtpgrey-02.html

is designed to communicate the optimal retry time to clients to help 
them minimize the overhead associated with blind server temporary 
rejections which puts pressure on the client-side requeuing strategies.

I think the general belief among our peers during the development of 
this I-D has been that we should really look for a new and specific 
reply code that raises the bar for newer C/S negotiations to take place 
with backward compatibility.   SMTPGREY attempts to leverage a formal 
text response syntax to provide extended response information.  For example:

Single line responses:

   450 4.7.1. Greylist enabled. retry=00:02:00
   451 Temporary rejection. retry=00:00:30
   450 4.7.1. Temporary Greylist rejection. retry=01-00:10:00
   451 TempFail Retry=00:00:55
   421 Your connection is greylisted. Please try again later 
(retry=00:01:00)

Multiple lines response:

   451-Greylisted. See policy http://example.com/GreyList-Policy
   451 Retry=00:02:00

A proof of concept has been shown that this actually works to improve 
the communications with MTA and minimizes their delays and retries to 
just 2 attempts.

> negotiation.  SPF is not about authentication.  SPF is an authorization

I can interpret SPF as an authentication protocol for IP::DOMAIN 
association authentication assertion which allow for the policy-based 
domain defined authorization for the sender domain to send mail.   This 
is can be done at the SMTP level prior to the DATA state.  DKIM is a 
Payload (RFC 822/2822/5322) protocol which would require the DATA state 
to be reached first in order to apply any sort of dynamic SMTP online 
response other than 250 accept.

> this if anyone is interested.

Having a hard time understanding how IPv6 has anything to do with DKIM 
or SPF that would be different than IPv4.  Even then, why make the 
distinction?  Does it matter what port is used?  The public vs private 
port?    Only with private port can you raise the bar for client 
correction (SMTP compliancy) over what is currently allowed for public 
port operations where there is legacy relaxations to allow for anonymous 
local or hosted domain reception of mail. Not open relay.

I am getting the idea that you focusing on IPv6 as the trigger to raise 
the bar for anything that is currently optional - DKIM, SPF and any 
other current augmented email technology.

How do you separate this with IPv4 design requirements that still needs 
to the in place, and most likely forever?

--
HLS

---

From: Douglas Otis <[email-masked]>
Date: Thu, 28 Mar 2013 20:29:14 -0700

Hello Hector,

On Mar 28, 2013, at 3:53 PM, Hector Santos <[email-masked]> wrote:

> Whats the difference with IPv6 connections?  Should it matter? Does it matter?

IPv6 makes publishing IP address reputations impractical.  Since IP address reputation has been a primary method for identifying abusive sources with IPv4, imposing ineffective and flaky replacement strategies has an effect of deterring IPv6 use.

> This requires a sign up to obtain/view. Sorry.

Javascript disabled?  Here is a simpler alternative:
http://www.bungi.com/Dom-v6.pdf

> Don't follow. A 55x is a permanent rejection. Not a SMTP protocol instruction to retry.

The 554 return code is to indicate no SMTP service is at the host.  It is illogical to assume other MX records will offer different results, but that is the expectation for their pseudo authentication scheme to work. 

> It may be, but it may force a client to continue a AUTH sequence and thats not possible if ESMTP is not in place or AUTH is not part of the ESMTP options.

Improved authentications using a negotiation sequence offers efficient and robust means for ensuring email delivery integrity.  DKIM can not control abusive sources.  SPF does not offer source authentication for mitigation control either.   

> Don't follow.  SMTP clients are following a SMTP state machine:

Some providers are trying to make an exception for 554 to mean the host does not support SMTP (when DKIM or SPF was not previously seen from a specific client). This is to imply a need to keep trying other hosts in other MX records after retrying a 421 code.   WIth IPv6, the same providers will also impose a flaky reverse DNS PTR record requirement as well.   For IPv4, these PTR records helped in the exclusion of residential access.   For IPv6, such mapping is also impractical and represent a waste of resources.

> I can interpret SPF as an authentication protocol for IP::DOMAIN association authentication assertion which allow for the policy-based domain defined authorization for the sender domain to send mail.

Many domains are obligated to authorize outbound servers being shared by other domains.  Assumptions that authorization means authentication may prove damaging by holding the wrong domains accountable.   Nothing ensures the return path has been asserted by the return path domain.

>   This is can be done at the SMTP level prior to the DATA state.  DKIM is a Payload (RFC 822/2822/5322) protocol which would require the DATA state to be reached first in order to apply any sort of dynamic SMTP online response other than 250 accept.

Aren't SMTP Auth negotiations the right methodology?

> Having a hard time understanding how IPv6 has anything to do with DKIM or SPF that would be different than IPv4.  Even then, why make the distinction?  Does it matter what port is used?  The public vs private port?    Only with private port can you raise the bar for client correction (SMTP compliancy) over what is currently allowed for public port operations where there is legacy relaxations to allow for anonymous local or hosted domain reception of mail. Not open relay.
> 
> I am getting the idea that you focusing on IPv6 as the trigger to raise the bar for anything that is currently optional - DKIM, SPF and any other current augmented email technology.
> 
> How do you separate this with IPv4 design requirements that still needs to the in place, and most likely forever?

IPv6 makes publishing IP address reputations impractical and why some are attempting a kludge that falls short of authenticating the source or at being reliable.

Regards,
Douglas Otis

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 29 Mar 2013 09:13:45 +0100 (CET)

On Thu, 28 Mar 2013, Douglas Otis wrote:

> IPv6 makes publishing IP address reputations impractical.  Since IP 
> address reputation has been a primary method for identifying abusive 
> sources with IPv4, imposing ineffective and flaky replacement strategies 
> has an effect of deterring IPv6 use.

My belief is that IP address reputation has always been flakey, it's just 
vastly more so with IPv6.

What we need is a way to identify a "entity" subnet size. This work is 
probably wasted on IPv4, but it's definitely needed for IPv6. The ISP in 
question needs to be able to publish customer/entity subnet size so 
reputation can be done at this level.

This information might today be available using whois to the RIR, but 
that's not very practical publication method for quick lookups.

---

From: John Curran <[email-masked]>
Date: Fri, 29 Mar 2013 07:28:13 -0400

On Mar 29, 2013, at 4:13 AM, Mikael Abrahamsson <[email-masked]> wrote:

> My belief is that IP address reputation has always been flakey, it's just vastly more so with IPv6.
> 
> What we need is a way to identify a "entity" subnet size. This work is probably wasted on IPv4, but it's definitely needed for IPv6. The ISP in question needs to be able to publish customer/entity subnet size so reputation can be done at this level.

This approach works fine if one presumes that the problem is always just
the customer (i.e. their ISP is actively interested in helping solve the 
problem.)  For ISPs who are not as interested (or may have an actual 
motivation to hinder resolution of the problem), this will not work.  

While the above situation has also been somewhat true with IPv4, it is 
definitely the case with IPv6, since the typical address space allocation 
sizes provide ample space for whitewashing customers into new prefixes.  
As a result, it is questionable whether any IPv6 address-based reputation 
system can be successful (at least those based on voluntary principles.)

/John

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 29 Mar 2013 12:38:34 +0100 (CET)

> motivation to hinder resolution of the problem), this will not work.

Well, I would also like to see reputation done on per-ISP level. If an ISP 
doesn't care, then the reputation of all the customers behind that ISP is 
lower.

> definitely the case with IPv6, since the typical address space 
> allocation sizes provide ample space for whitewashing customers into new 
> prefixes. As a result, it is questionable whether any IPv6 address-based 
> reputation system can be successful (at least those based on voluntary 
> principles.)

This is absolutely a problem. I encourage all ISPs to give customers the 
same addresses all the time, and publish if they provide dynamic. This is 
one more factor which should be included in the publication 
(static/dynamic allocation of addresses). So basically dynamic ones should 
be treated like "dialup space" today, static ones can actually be trusted 
if the ISP is reliable. If static and reliable ISP = reputation of one 
customer of allocation size can be blacklisted without affecting other 
customers.

ISPs that do this reliably should have high reputation, and the ones who 
don't, should get low reputation. Low reputation ISPs I guess none of 
this data should be trusted.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: "John Levine" <[email-masked]>
Date: 29 Mar 2013 16:58:43 -0000

>system can be successful (at least those based on voluntary principles.)

It can probably work for whitelisting well behaved senders, give or take
the DNS cache busting issues of IPv6 per-message lookups.

Since a bad guy can easily hop to a new IP for every message (offering
interesting new frontiers in listwashing) I agree that it's a losing
battle for blacklisting, other than blocking large ranges of hostile
networks.

Fortunately, the IETF as a whole is not called upon to solve this
problem right now.  People interested in mail reputation are welcome
to drop by the spfbis WG and the discussions in appsarea about
updating authentication and authentication logging RFCs.

---

From: Douglas Otis <[email-masked]>
Date: Fri, 29 Mar 2013 10:28:25 -0700

> updating authentication and authentication logging RFCs.

Dear John,

The Internet is under a DDoS attack specifically against an email address reputation service.  This affects everyone, especially the IETF.

Strategies not premised on low overhead AUTHENTICATION are of little benefit.   We can no longer continue business as usual.  I call upon the IETF to solve this problem.  It is within their charter.  It is within their capabilities.  We can not make everyone upgrade, but we can establish a path that has a chance of offering a solution.

Regards,
Douglas Otis

---

From: Doug Barton <[email-masked]>
Date: Fri, 29 Mar 2013 13:57:04 -0700

On 03/28/2013 08:29 PM, Douglas Otis wrote:
> IPv6 makes publishing IP address reputations impractical.

For individual addresses, sure. But one of the (if not *the*) primary 
benefits of v4 reputation is the test of whether or not the address is 
in a botnet range (aka, ranges assigned to end-users). That will still 
work quite nicely with IPv6, assuming that the ISPs cooperate at roughly 
the same levels they do now with IPv4.

> Since IP

I personally don't believe this is true. One of the things that IPv6 
advocates encourage folks to do is to put their mail infrastructure on 
IPv6 first as a test, since the mail protocol is relatively forgiving. 
That is a good piece of advice, which a lot of sites seem to have followed.

That said, I don't necessarily disagree with your thoughts about domain 
reputation vs. IP address reputation. I think that there is room for 
discussion about that.

On the other hand I don't agree with your negative view about DKIM and 
SPF. I recently moved from an IHP to my own VPS, and set up both DKIM 
and SPF for my active domains. The former has allowed me to send mail to 
various places that didn't accept mail from my old IHP's servers. And I 
have a hard-fail on my SPF records, and that has cut to nearly zero the 
amount of Joe-job backscatter I receive (whereas previously it was in 
the 3-10 messages per day range).

I don't think either mechanism is perfect, and I'm intrigued by DMARC 
although I haven't really had time to study it yet. But in my anecdotal 
experience both DKIM and SPF are useful at least, and seem to work well, 
so I'd hate to see them out with the bathwater.

Doug

---

From: Hector Santos <[email-masked]>
Date: Sat, 30 Mar 2013 04:41:47 -0400

Hi Doug,

This sounds urgent.  I am not seeing this urgency, but maybe we just 
have it under control.

Another side question Doug, is this an application-level based 
filtering?  Can one be authenticated lets say for SMTP but not WEB? 
Is the filtering applied across all protocols? Is it the IP or mail level?

I see interesting scenarios when good guys who actually fail to connect 
and authenticate via one protocol and when a limit is reached, the IP is 
(optionally) blocked across all other entry points as well for a limited 
period.  Its all done automatically, they forget they need to wait for 5 
minutes!  Its works great auto-protecting them against "bad guys."

--
HLS

---

From: "Livingood, Jason" <[email-masked]>
Date: Sat, 30 Mar 2013 14:57:18 +0000

On 3/29/13 12:58 PM, "John Levine" <[email-masked]> wrote:

>>As a result, it is questionable whether any IPv6 address-based

Agree. The IP blacklisting that worked well for IPv4 is completely
unsuited for IPv6 (I'd go as far as to say it is a complete failure, no
matter if you look at different size prefixes or not).

The only model that I personally can see working at the moment for IPv6 is
a mix of domain-based reputation and whitelisting. I like domain-based
better since it is managed by sending domains on a distributed basis.

Mail acceptance for IPv4 worked inclusively - receivers accept unless IP
reputation or other factors failed. IMHO with IPv6 that model may need to
be turned around to an exclusive one - so receivers will not accept mail
unless certain factors are met (like domain-based authentication or the
IPv6 address is on a whitelist). I'd expect MAAWG will continue to be a
good place for mail ops folks to work through this stuff.

- Jason

---

From: John C Klensin <[email-masked]>
Date: Sat, 30 Mar 2013 11:43:59 -0400

--On Saturday, March 30, 2013 14:57 +0000 "Livingood, Jason"
<[email-masked]> wrote:

>...
> Mail acceptance for IPv4 worked inclusively - receivers accept
> unless IP reputation or other factors failed. IMHO with IPv6
> that model may need to be turned around to an exclusive one -
> so receivers will not accept mail unless certain factors are
> met (like domain-based authentication or the IPv6 address is
> on a whitelist). I'd expect MAAWG will continue to be a good
> place for mail ops folks to work through this stuff.

Without expressing any particular opinion about the above, I
would encourage people to remember that one of the fundamental
design decisions about Internet mail --predating even RFCs
821/822-- has been that messages will either be delivered or
explicitly rejected in a way that produces an NDN to the sender.
There was obviously an exception case when the NDN could not be
delivered, but it was, and remains, rare for
properly-constructed legitimate messages.  Put differently, it
is not a "sometimes works" or "best efforts" service: the sender
has the right to assume that silence implies successful delivery.

Legitimate concerns about "Joe-job" attacks, blowback, and
similar nonsense have created good operational reasons to not
have every undeliverable message generate an NDN.   From the
point of view of a legitimate sender, silence no longer reliably
indicates delivery: it could indicate delivery, silent dropping
of the message for a reason indiscernible to the sender, or. in
rare cases, loss of the NDN.

We specified delivery notifications to permit a sender to get a
higher level of assurance about delivery, but support for them
has always been optional.  If a sender asks for a delivery
notification, no response (and no NDN) can mean that the
delivery notification got lost, the NDN, got lost, or the
message was successfully delivered but delivery notifications
were not supported by the delivery MTA -- a rather ambiguous
situation.

It sometimes feels as if anti-spam efforts are trending in the
direction of its being acceptable to accidentally discard a few
dozen legitimate messages if doing so allows blocking a few
thousand unsolicited/undesired ones.   I hope we never consider
that a good tradeoff but, if we do, the decisions should at
least be made openly and with some degree of community
consensus.  

If we are going to take further steps in the direction of
silently dropping undeliverable or undesired messages, I believe
we need to carefully consider the implications of that change to
the email model and, in particular, whether support for delivery
notifications should be made mandatory or other changes are
needed to keep email as a much more reliable service than, e.g.,
putting messages into bottles and throwing the bottles into
large bodies of water.   We should probably also encourage
requirements that email vendors and providers who consider
discarding possibly-legitimate messages reasonable in order to
reduce spam to disclose that fact to their users, customers, and
the community.

best,
    john

---

From: Douglas Otis <[email-masked]>
Date: Sat, 30 Mar 2013 15:46:13 -0700

Dear Jason,

> matter if you look at different size prefixes or not).

Agreed.

> better since it is managed by sending domains on a distributed basis.

Current domain based strategies such as SPF offer fragile dependence on return path parameters that may incur a large number of transactions to resolve authorizations.  Use of DKIM must also consider the signing domain neither controls actual sources, intended recipients, or message relaying.

> good place for mail ops folks to work through this stuff.

While SPF offered a fix for DSN back-scatter, neither this scheme nor DKIM provide a suitable basis for domain reputation.  Neither authorization nor signed message content provide any direct evidence of abuse accountability.  

Permission for this occurs by leaving the future of email primarily in the hands of those having conflicts of interest.  For example, none of the current domain based schemes offer a means to hold those paid to send bulk email accountable.  Several would even be happy to see IPv6 email require IPv4 providers to relay IPv6 email.

Here is the link that illustrates the serious problem.
http://www.bungi.com/Dom-v6.pdf

And again, I call on the IETF to work on this problem.

Regards,
Douglas Otis

---

From: Dave Crocker <[email-masked]>
Date: Sat, 30 Mar 2013 16:46:05 -0700

> good place for mail ops folks to work through this stuff.

Jason (et al),

You are echoing views that have been acquiring quite a few supporters 
over the last year or so.  While entirely understandable, they face a 
number of challenges, not the least being serious dependency on the 
unstated particulars.  That is, a broad statement of principle, such as 
you proffer, isn't enough for evaluating either efficacy of the approach 
or its drawbacks; there need to be very concrete details specified, with 
careful consideration of the way they will affect /legitimate/ Internet 
mail.[1]

One issue does not require that detail:

      Specifically, the way v6 mail is discussed assumes that it is 
different from -- independent of -- v4 mail.  It isn't and it won't be. 
  It's an extension of the existing Internet Mail service, which is 
largely ignorant of the underlying IP layer.

      I realize that's not true of the complex protection engines that 
have been developed in the face of abuse; but my point is that the 
functionality seen by users neither knows nor cares about transport and 
below.

Folks might not realize just how multi-protocol Internet Mail transport 
used to be.[2]  Having mail running in parallel over two 'transport' 
systems will merely be a visit back to that.  Or, at least, it should be.

To the extent that the basic user experience is different, depending 
upon the lower layer being used, Internet Mail is going to be even 
messier for those users than it is now.

It's my understanding that messier user experience translates into 
higher customer support costs...

d/

[1] Given that the default-deny approach has been voiced for some time 
now, the lack of specifications, including how to integrate it with 
existing mail, should start to trouble us all.

[2] To Be "On" the Internet, RFC 1775
-- 
  Dave Crocker
  Brandenburg InternetWorking
  bbiw.net

---

From: SM <[email-masked]>
Date: Sat, 30 Mar 2013 17:31:41 -0700

>IPv6 address is on a whitelist). I'd expect MAAWG will continue to be a

Some receivers currently rely on DNSBLs to decide for them whether to 
accept a message or not.  A number of these DNSBLs only offer support 
for IPv4 addresses.  I assume that these DNSBLs do not know how to 
handle IPv6 addresses.  Changing the email model to one where there 
is prior agreement or else the receiver will reject the message 
ignores one of the fundamentals which has made email a success.

At 15:46 30-03-2013, Douglas Otis wrote:
>Current domain based strategies such as SPF offer fragile dependence 
>on return path parameters that may incur a large number of 
>transactions to resolve authorizations.  Use

That is how SPF works.  I presume that persons who are commenting 
about SPF will be reviewing the updated version of the specification 
to see whether it works like that or not.

>Permission for this occurs by leaving the future of email primarily 
>in the hands of those having conflicts of interest.  For example, 
>none of the current domain based schemes

There are conflicts of interest on every side.  Things usually do not 
work well when people cannot make a gesture of good faith.

>And again, I call on the IETF to work on this problem.

That sounds like a call for individuals who are easily cheated, 
deceived, or imposed upon to work on the problem. :-)

Regards,
-sm

---

From: Christian Huitema <[email-masked]>
Date: Sun, 31 Mar 2013 06:26:29 +0000

> IPv6 makes publishing IP address reputations impractical.  Since IP address reputation has been a primary method for identifying abusive sources with IPv4, imposing ineffective and flaky > replacement strategies has an effect of deterring IPv6 use. 

In practice, the /64 prefix of the IPv6 address has very much the same "administrative" properties as the /32 value of the IPv4 address. It should be fairly straightforward to update a reputation system to manage the /64 prefixes of IPv6. This seems somewhat more practical than trying to change the behavior of mail agent if their connectivity happens to use IPv6.

-- Christian Huitema

---

From: Doug Barton <[email-masked]>
Date: Sun, 31 Mar 2013 01:23:59 -0700

> In practice, the /64 prefix of the IPv6 address has very much the same "administrative" properties as the /32 value of the IPv4 address. It should be fairly straightforward to update a reputation s…

That only works insofar as the provider does not follow the standard 
recommendation to issue a /48. If they do, the abuser has 65k /64s to 
operate in.

What's needed is a little more intelligence about how the networks which 
the IPv6 addresses are located are structured. Similar to the way that 
reputation lists nowadays will black list a whole /24 if 1 or a few 
addresses within it send spam.

The problems are not insoluble, they're just different, and arguably 
more complex in v6. It's also likely that in the end more work on 
reputation lists will provide less benefit than it did in the v4 world. 
But that's the world we live in now.

Doug

---

From: "John Levine" <[email-masked]>
Date: 31 Mar 2013 18:55:21 -0000

>In practice, the /64 prefix of the IPv6 address has very much the same
>"administrative" properties as the /32 value of the IPv4 address.

You would hope so, but I know hosting places that give their customers
a /128 in a shared /64.  They claim that their routers make this hard
to fix.  I don't know the details.

Like I said, whitelisting known good single IPs can probably work, but
it is a major unsolved question how to figure out the size of the
relevant bad range if you see an IP doing bad stuff.

R's,
John

PS: Just to save time, any answer starting "all you need to do is ..."
doesn't work.  We've been around this barn enough times that there is
severe soil erosion.

---

From: Hector Santos <[email-masked]>
Date: Sun, 31 Mar 2013 18:17:37 -0400

Good points Dave.

However, I would suggest that having tighter controls on the transport 
practice, e.g.; SMTP handshaking compliancy, following and honoring 
exclusive domain published policies, does help minimize support cost.

--
HLS