Subject: [Asrg] How will we manage IPv6 spam?

From: "John R. Levine" <[email-masked]>
Date: 17 Aug 2012 16:22:55 -0400

Hi.  Remember the ASRG?  I was hoping it might do a little research.

In talking to people about IPv6 mail, I'm still coming to the conclusion 
that anyone who thinks they know how they're going to handle it, beyond 
the current toy scale, doesn't understand the problem.  Things we might 
address include:

* Will DNSBLs that work like v4 BLs, with a query per IP, blow out DNS 
caches?  If so, can this be solved by hacks like partitioned caches that 
treat BL traffic separately?  Would something like my B-tree hack work 
better?

* Is there some reasonable way for networks to publish allocation 
granularity, e.g., this range is a /64 per user, that range is individual 
hosts?  If they can, how useful would it be to running BLs or otherwise 
making filtering decisions?

* Is there a practical way to do hard or soft whitelisting of V6 mail 
hosts?  (Hard: no body filtering, soft: with body filtering)

* Should large and small systems use the same filtering techniques?  Large 
systems have larger mail volume and so can build better models of incoming 
traffic, small systems can afford cruder filters like no mail from Korea

* Can we build models to predict this stuff now, since under the most 
optimistic scenario it'll still be years before the v6 mail volume 
approaches v4 mail volume.

R's,
John

---

From: Daniel Feenberg <[email-masked]>
Date: Fri, 17 Aug 2012 16:51:14 -0400 (EDT)

On Fri, 17 Aug 2012, John R. Levine wrote:

> In talking to people about IPv6 mail, I'm still coming to the conclusion that 
> anyone who thinks they know how they're going to handle it, beyond the 
> current toy scale, doesn't understand the problem.  Things we might address 
> include:
>

I would not expect to accept any IPv6 mail until users come forward to 
show me that they wish to correspond with MTAs that have no IPv4 
connection ability. While this may happen in the fullness of time, I don't 
expect it soon. Now and for the foreseeable future such a system would 
have very little connectivity, far less than a blacklisted spam source.

I understand that a non-trivial number of MTAs are currently able to use 
IPv6, but I also understand that all of the ones we wish to correspond 
with are also capable of using IPv4. Since our anti-spam technology 
(Spamhaus) is very dependent on IPv4 blacklists, it would be very 
undesirable to add a general IPv6 port. I would add a handful of IPv6 only 
sites to a whitelist or subscribe to a responsible whitelist long before 
attempting to accept messages from unknown IPv6 mtas.

We are all familiar with the concept that you "can't enumerate badness". 
The IPv4 address space is small enough that you actually can, and it
has worked very well for us. In IPv6, I don't think it would work at all.

dan feenberg
NBER

---

From: Michael Thomas <[email-masked]>
Date: Fri, 17 Aug 2012 14:04:33 -0700

On 08/17/2012 01:51 PM, Daniel Feenberg wrote:
>> In talking to people about IPv6 mail, I'm still coming to the conclusion that anyone who thinks they know how they're going to handle it, beyond the current toy scale, doesn't understand the problem.  Things we might address include:
>>
>
> I would not expect to accept any IPv6 mail until users come forward to show me that they wish to correspond with MTAs that have no IPv4 connection ability. While this may happen in the fullness of time, I don't expect it soon. Now and for the foreseeable future such a system would have very little connectivity, far less than a blacklisted spam source.

Host operating systems -- all of them to my knowledge -- prefer v6 over
v4 if you have a public v6 address. So the mere existence of a AAAA associated
with the MX will cause the sender to pick the v6 destination. I have a v6 mail
system and got bitten because I had forgot to put up the v6 reverse map. It
will happen just as a natural consequence of people enabling v6 on their
infrastructure.

Mike

---

From: Daniel Feenberg <[email-masked]>
Date: Fri, 17 Aug 2012 17:08:02 -0400 (EDT)

On Fri, 17 Aug 2012, Michael Thomas wrote:

>> I would not expect to accept any IPv6 mail until users come forward to show 
>> me that they wish to correspond with MTAs that have no IPv4 connection 
>> ability. While this may happen in the fullness of time, I don't expect it 
>> soon. Now and for the foreseeable future such a system would have very 
>> little connectivity, far less than a blacklisted spam source.
>
> v4 if you have a public v6 address. So the mere existence of a AAAA 
> associated with the MX will cause the sender to pick the v6 destination. 
> I have a v6 mail system and got bitten because I had forgot to put up 
> the v6 reverse map. It will happen just as a natural consequence of 
> people enabling v6 on their infrastructure.

This sounds inconvenient. If I want to accept mail from one IPv6 host, 
then all the IPv6 hosts will want to use IPv6, and unless I accept mail 
from unknown IPv6 hosts, mail from hosts that would have been accepted 
over IPv4 will be rejected? That is a problem, but I can't imagine IPv6 
acceptance being so universal that there would be more desirable mail from 
IPv6 only hosts than from IPv4 only hosts, at least not for some decades. 
This is especially true since more important hosts are more likely to have 
access to IPv4 addresses. I actually wonder if the transition could ever 
occur as long as IPv4 is supported at all by ISPs.

daniel feenberg
NBER

---

From: Paul Smith <[email-masked]>
Date: Sat, 18 Aug 2012 01:02:07 +0100

On 17/08/2012 22:08, Daniel Feenberg wrote:
>> Host operating systems -- all of them to my knowledge -- prefer v6 
>> over v4 if you have a public v6 address. So the mere existence of a 
>> AAAA associated with the MX will cause the sender to pick the v6 
>> destination. I have a v6 mail system and got bitten because I had 
>> forgot to put up the v6 reverse map. It will happen just as a natural 
>> consequence of people enabling v6 on their infrastructure.
>
> then all the IPv6 hosts will want to use IPv6, and unless I accept 
> mail from unknown IPv6 hosts, mail from hosts that would have been 
> accepted over IPv4 will be rejected? 
This is a fair point. Given that IPv6 is rare at the moment and is 
really an 'extension' to SMTP anyway, maybe we should be looking at a 
further extension which allows an SMTP receiver to say 'retry on my IPv4 
address'

However, it is up to the sending MTA whether it uses an IPv4 or IPv6 
address. The 'OS may prefer IPv6', but the MTA could get all the 
available IP addresses for the MX name, then chooses which to use. If 
the OS has a 'connect to name' function then it may just connect to the 
IPv6 address, but if you have to use 'gethostbyname' then 'connect to IP 
address' functions then you can choose which results of the 
'gethostbyname' function to use, and which to ignore, and which order to 
try them in.

That's what ours does, and it can be configured to ignore the IPv6 
addresses and just try to connect to the IPv4 ones, or to try IPv4 
before IPv6.

> This is especially true since more important hosts are more likely to 
> have access to IPv4 addresses. I actually wonder if the transition 
> could ever occur as long as IPv4 is supported at all by ISPs.

This is a fair point as well, and I do wonder that as well. We added 
IPv6 support to our MTA as a 'selling point', but I'd generally 
recommend users not to enable it.

I suspect that if IPv6 did become widely used for mail, then we'd have 
to start moving towards a system of 'trusted MTAs' to try to limit the 
spam issue. If we're going to do that, then there would be a limited 
number of those, so why not have those 'trusted hosts' use IPv4? Then 
any IPv6 SMTP would just be between mutually agreed upon MTAs, with the 
'edge' MTA using IPv4 to talk to other 'unknown' MTAs. (hope that makes 
sense)

-

Paul Smith Computer Services
Tel: 01484 855800
Vat No: GB 685 6987 53

---

From: Michael Thomas <[email-masked]>
Date: Fri, 17 Aug 2012 17:21:30 -0700

On 08/17/2012 05:02 PM, Paul Smith wrote:
>>> Host operating systems -- all of them to my knowledge -- prefer v6 over v4 if you have a public v6 address. So the mere existence of a AAAA associated with the MX will cause the sender to pick the v6 destination. I have a v6 mail system and got bitten because I had forgot to put up the v6 reverse map. It will happen just as a natural consequence of people enabling v6 on their infrastructure.
>>
>> This sounds inconvenient. If I want to accept mail from one IPv6 host, then all the IPv6 hosts will want to use IPv6, and unless I accept mail from unknown IPv6 hosts, mail from hosts that would have been accepted over IPv4 will be rejected? 
> This is a fair point. Given that IPv6 is rare at the moment and is really an 'extension' to SMTP anyway, maybe we should be looking at a further extension which allows an SMTP receiver to say 'retry on my IPv4 address'

Seriously, let's not go here. Any such extension would be DOA in the IETF anyway.

If DNSBL's are the reason you would want to stay in v4 land, I'd suggest
the problem is with the way DNSBL's are engineered, not v6. Thislist  being
an IETF research group should just accept that v6 will arrive and not try
to roll back time.

Mike

---

From: "John Levine" <[email-masked]>
Date: 18 Aug 2012 01:40:25 -0000

>> I would not expect to accept any IPv6 mail until users come forward to show me that
>they wish to correspond with MTAs that have no IPv4 connection ability. While this may
>happen in the fullness of time, I don't expect it soon. Now and for the foreseeable
>future such a system would have very little connectivity, far less than a blacklisted
>spam source.

I agree that it will be a while before there is any worthwhile mail
that can't be delivered over IPv4.  But Comcast and Gmail already do
mail over IPv6, and other big providers are likely to add it this
year.  If you have any v6 mail connectivity at all, you will start
to get real mail over it.

So it would be nice, what with the ASRG's charter to do, you know,
research, if we started thinking now about how to manage spam over
V6 rather than sticking our fingers in our ears.

R's,
John

---

From: Daniel Feenberg <[email-masked]>
Date: Sat, 18 Aug 2012 06:50:02 -0400 (EDT)

On Fri, 17 Aug 2012, Michael Thomas wrote:

>>>> I have a v6 mail system and got bitten because I had forgot to put up the 
>>>> v6 reverse map. It will happen just as a natural consequence of people 
>>>> enabling v6 on their infrastructure.
>>> 
>>> over IPv4 will be rejected? 
>> This is a fair point. Given that IPv6 is rare at the moment and is really 
>> an 'extension' to SMTP anyway, maybe we should be looking at a further 
>> extension which allows an SMTP receiver to say 'retry on my IPv4 address'
>

How would you engineer a DNSBL for IPv6?

> an IETF research group should just accept that v6 will arrive and not try

The question isn't so much when IPv6 will arrive, there are lots of uses 
for IPv6, but when IPv4 will depart. And I don't see any incentive for
our correspondents to establish IPv6 mtas when IPv4 mtas have much
greater acceptability to receivers. That is independent of the DNSBL
situation but the lack of an effective DNSBL further encourages mtas
not to accept IPv6 messages, a factor which strongly militates against
a gradual transition.

You could argue that new institutions will not be able to obtain IPv4
addresses, and therefore will be forced to adopt IPv6 for their mta. I 
expect any significant email source will find an IPv4 address for their
mta, even if the rest of their servers are IPv6 only. The transition for
other services is much easier, since in other cases offering a dual
stack has no downside. It is only in the case of the mta that a dual
stack results in a significant lose of service. Therefore, no transition
without a flag day, and flag days are very difficult to arrange.

Another possibility is that an IPv6 DNSBL could be established, but
listing very broad address ranges, such as /32s. I don't know how
that would work out, but I would expect a lot of colateral damage from 
such a broad listing.

daniel feenberg

>
> Mike
> _______________________________________________
> Asrg mailing list
> [email-masked]
> http://www.irtf.org/mailman/listinfo/asrg
>

---

From: Daniel Feenberg <[email-masked]>
Date: Sat, 18 Aug 2012 06:59:20 -0400 (EDT)

> to get real mail over it.

Yes, but I only have an incentive to have IPv6 mail connectivity
if I would otherwise lose email. And I have heard of no sender
that doesn't fall back to IPv4.

> V6 rather than sticking our fingers in our ears.

It is one thing to say that support for IPv6 is the morally correct
action, and another to point out a benefit to the receiver of
accepting IPv6 mail, when all of the same mail is available over
IPv4. Will it be possible for the IETF to convince major legitimate
senders to drop support for IPv4 mail? Would anyone here drop support
of IPv4 email for their employer or customer as a matter of
principle?

daniel feenberg

---

From: Paul Smith <[email-masked]>
Date: Sun, 19 Aug 2012 13:53:52 +0100

> of IPv4 email for their employer or customer as a matter of

+1

I can't see IPv4 SMTP stopping until IPv4 is obsolete. Even if a 
business can only get IPv6 addresses for themselves, there will be 
people offering IPv4 SMTP gateways that they can use, and they'll use 
those gateways or they'll be cut off from most of the world who will 
still be using IPv4. So, IPv4 mail is pretty much here to stay for the 
foreseeable future. So, given that, why would people start accepting 
IPv6 mail? It may not be the politically correct thing to say, but 
that's the way it is...

However, discussing 'is there a better way of handling spam than RBLs?' 
is worthwhile with IPv4 regardless of whether IPv6 is in use or not.

Personally, I think SMTP as we know it is more likely to be killed off 
by spam than it is to become reliant on IPv6.

My view is that we need to move towards having 'authorised senders', 
possibly by buying a a certificate (or similar) with a 'CA' checking 
you're 'suitable' to be sending mail (eg a legitimate company or 
person), and able to revoke the certificate if you abuse it. Yes, there 
would be a cost to this (if you want to be able to send mail from your 
domain - I'm thinking similar costs to, say, a code-signing certificate 
which has more stringent checks than a website SSL cert), so it wouldn't 
be popular, but I'm beginning to think that would be one of the only 
ways which would work and wouldn't be a kludge as RBLs or greylisting 
etc are.

I'd have thought this could be added without that much difficulty by 
requiring TLS for all public MTA-MTA transactions and starting to do 
client certificate based authentication (any non TLS or non-certified 
transactions could be accepted, but treated as suspect and scanned more 
thoroughly)

While SPF and DKIM etc should in theory help, they seem to be unpopular 
to some people, and while they can be useful in the fight against spam, 
they aren't actually anti-spam systems themselves.

-

---

From: "Emanuele Balla (aka Skull)" <[email-masked]>
Date: Sun, 19 Aug 2012 16:13:31 +0200

> treat BL traffic separately?  Would something like my B-tree hack work

If this problem is going to raise, it's going to raise exactly the same
way with rDNS as well, so having v6 DNSBLs in place is going to make the
problem worse by just a factor related only with the number of DNSBLs in
place. 2x? 5x?

This is a small factor, compared to the size of the problem itself.

FWIW, the DNSBL case can be worked around using 0 as TTL for DNSBLs
(directly on the DNSBL side, or on the caching side for resolvers with
the ability to do that). This is going to increase the load to DNSBL
mirrors, but that would probably be much more easily worked with than
the cache blowout problem.

Another chance is to move part of the logic into the lookup tool and
make it query for the /64 network instead of the full address, and run
DNSBLs accordingly.

This would significantly reduce the size of the caching problem, but
would render listings much less granular and whiltelisting of single
hosts basically impossible...

> granularity, e.g., this range is a /64 per user, that range is
> individual hosts?  If they can, how useful would it be to running BLs or
> otherwise making filtering decisions?

I already had the chance to tell you that, but RIPE DB provides an
"assignment-size" field with this explicit purpose:

inet6num:       2a02:09a8:FF00::/40
netname:        SPIN-IT-IPv6-SOHO-Trieste1
descr:          IPv6 SOHO residentials
country:        IT
admin-c:        AL6557-RIPE
tech-c:         SPIN3-RIPE
mnt-irt:        IRT-AS6734
status:         AGGREGATED-BY-LIR
assignment-size: 56
mnt-by:         AS6734-MNT
source:         RIPE # Filtered

IMHO encouraging a similar setup won't hurt.

> hosts?  (Hard: no body filtering, soft: with body filtering)

IMHO no real differences from IPv4 except those already cited above at
point 1.

> * Should large and small systems use the same filtering techniques? 
> Large systems have larger mail volume and so can build better models of
> incoming traffic, small systems can afford cruder filters like no mail
> from Korea

IMHO there's no real difference with IPv4 here.
Except that large systems will probably rely on local mirrors (and
sometimes on technologies that do not involve DNS at all) hence they'll
be less impacted by the caching problem: they can run everything with
TTL set to 0 without heavily impacting lookup latencies.
The same is not necessarily true for small systems.

> approaches v4 mail volume.

DUNNO

---

From: SM <[email-masked]>
Date: Sun, 19 Aug 2012 12:52:03 -0700

>accepting IPv6 mail, when all of the same mail is available over

There isn't any morally correct action.

>senders to drop support for IPv4 mail? Would anyone here drop support

No.

>of IPv4 email for their employer or customer as a matter of

No.

The subject line is about how a problem can be tackled.  Doing 
nothing is one of the options.

Regards,
-sm

---

From: "John Levine" <[email-masked]>
Date: 19 Aug 2012 23:38:36 -0000

>problem worse by just a factor related only with the number of DNSBLs in

That's true, but people I've talked to at large mail systems say
they're not planning to do rDNS lookups for v6 mail, both because of
the cache problems and because they don't think it will catch much
spam.

>the ability to do that).

That's true, but what would be really nice would be DNSBLs that tried
to be smart about TTLs based on the amount of traffic an IP sends.
I'd think it should be possible to estimate that pretty well from
query logs.

>make it query for the /64 network instead of the full address, ...

I think you'll also find that you're blacklisting whole racks at
hosting companies when one customer has an insecure PHP script.

>> granularity

>"assignment-size" field with this explicit purpose:

Do you really want people querying that at DNSBL rates?  This needs
to be at a lower level.

>> approaches v4 mail volume.

Hey, I know a research group where we could give it a try.

R's,
John

---

From: "Emanuele Balla (aka Skull)" <[email-masked]>
Date: Mon, 20 Aug 2012 11:08:33 +0200

> the cache problems and because they don't think it will catch much

Other than the spam problem there are other needs for rDNS lookups,
starting from logging...

So while I understand the approach (not that I completely agree with
it), but I'm not sure it's completely feasible unless we change a lot of
things in the way we usually work...

> I'd think it should be possible to estimate that pretty well from

Sure, that could be a way: starting every listing with 0 TTL and
increasing it up to a maximum if it's generating an intense mail flow.
Doing the opposite, though (lowering the TTL as the mail source "calms
down") could be less easy to do: the same presence of caching inhibits
the DNSBL from estimating the lookup rate...

Note anyway that we're only considering the case of positive DNS answers
(or listed entities), but I'd expect that most of the cache blowup
problem will be generated by NXDOMAINs, at list at first.
We have much less control on that...

> hosting companies when one customer has an insecure PHP script.

Know that, yes...

> Do you really want people querying that at DNSBL rates?  This needs

Not suggesting it for end-user lookups. Simply suggesting that if that
information is published in whois, it's there for whoever wants to
collect it and use it (starting with DNSBLs).

BTW, in order to make the same information available and easily
queryable from end users, the most obvious place to put it is probably
rDNS; but again, you'll have all the caching problems again.

> Hey, I know a research group where we could give it a try.

;-)

---

From: Matthias Leisi <[email-masked]>
Date: Mon, 20 Aug 2012 12:07:21 +0200

> We have much less control on that...

There may be slight differences if we consider the wider case of
DNSxLs in general, ie including whitelists which may have different
cache characteristics. At dnswl.org, we can generally live with TTLs
in the order of magnitude of days (although we make them shorter so
that we can remove/change entries reasonably fast).

At present, we have ~180k IPs listed (IPv4, there are only a handful
of IPv6 addresses which we do not yet publish due to the unclarity
about how this should be done). There are potentially another 100k or
so IPs which we do not list (yet) for various reasons (time to edit
them, questionable source, ...).

The data changes very slowly, most often somewhere between 50 and 250
changes per day (including score changes etc, not necessarily
removed/added IPs); some exceptional days may have 1'000 or 2'000
changes.

As long as operators of "legitimate" sending mail systems do not
change the distribution/IP patterns of their mailservers dramatically
in an IPv6 world (and there are few reasons why they should do so),
there is little impact on DNS infrastructure (auth + cache).

For the case of abusive behaviour (ie, spammer using vast pools of IP
addresses, eg > 10^3 IP addresses, either snowshoe-like or in cannon
style), DNS caches will only blow if receiving systems are "dumb", ie
do not handle dDoS-like patterns appropriately. That's already an
issue today, and will only be "somewhat worse" in an IPv6 world.

-- Matthias

---

From: "John Levine" <[email-masked]>
Date: 20 Aug 2012 16:51:10 -0000

>issue today, and will only be "somewhat worse" in an IPv6 world.

How can an MTA tell whether a connection from a hitherto unseen IP
is part of a DDoS or a real user sending mail for the first time?

R's,
John