Subject: [Asrg] DNSBL and IPv6

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 19 Oct 2012 08:22:27 +0200 (CEST)

Hello.

I just subscribed to this list and tried to read up.

I'm very interested in IPv6 and SPAM, and I'd like to add some to the two last 
threads here regarding DNSBL and IPv6.

Fundamentally in IPv6, a "customer" (or entity or whatever) will in a lot of 
cases not a single IP address, but a network.

Households will get /64s, or get a /56 via DHCPv6-PD. Phones get a /64 or a 
network via DHCPv6-PD. Companies get /48 (or something else, but a bunch of 
networks). This is fundamentally how IPv6 was intended to be used, and 
hopefully that's how most ISPs will deliver it to customers.

So for spam detection to happen, detection of what is a "customer" needs to 
happen, and this needs to be on a network level, not single IPv6 address level. 
The RIR databases (at least RIPE) contain information about what kind of 
per-customer subnet size is for a certain large block of addresses.

Equivalent in IPv4 is "this customer has an IPv4 /26" and spam blocking would 
be done on a per-customer level, not per unique IPv4 address.

I'm a routing guy, not MUA/MTA guy, so I have little insight in what things 
look like in the real world outside of my personal setup with postfix, some 
DNSBL and procmail/spamassassin.

What I feel needs to happen is that policy needs to put in place to RIRs (via 
ISPs) can present "what is a customer" on a network level, and then this 
information can be put into DNS somehow, and used for DNSBL.

Example:

An ISP serves 10000 households with connectivity, each household gets a /56, 
this is done via an IPv6 /42 (because this is in a single town and it's 
aggregated like that). So this /42 would be in some kind of "residential 
access" classification, so people could block on that, and if one wants to 
block unique spammers, then this needs to be identified on a /56 level.

In other places I've pitched that the RIRs would publish this information in 
some kind of format outside of whois, so perhaps we need to start there, to 
create a standard (I don't know who should create the standard, but agreeing 
that a standard is needed is one step) for how this information is published, 
is a first step.

Thoughts?

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Matthias Leisi <[email-masked]>
Date: Fri, 19 Oct 2012 08:52:12 +0200

> cases not a single IP address, but a network.

At the beginning of an SMTP transaction, all you have is a single IP.

> network via DHCPv6-PD. Companies get /48 (or something else, but a bunch of

As a spam filter (software developer), you may want to know more about
the reputation of the IP address connecting to you.

> happen, and this needs to be on a network level, not single IPv6 address
> level. The RIR databases (at least RIPE) contain information about what kind
> of per-customer subnet size is for a certain large block of addresses.

You need an algorithm where you start from a single IP address and
then potentially "move up" until you get a meaningful result. That's
more or less what the B-tree algorithm suggested by John Levine some
months ago would offer: variable "depth" and "density" of data
controlled by the DNSxL operator optimized for the (on average) lowest
number of lookups needed.

At the same time, having a standardised and light-weight protocol to
determine the allocation policy by the ISP would be hugely helpful
(this will likely then be mirrored by the DNSxL operator). Absent such
data,third parties have to fall back to some default /64 etc.

> What I feel needs to happen is that policy needs to put in place to RIRs
> (via ISPs) can present "what is a customer" on a network level, and then
> this information can be put into DNS somehow, and used for DNSBL.

I don't know whether RIRs can mandate the publication of this data
through policy.

-- Matthias

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 19 Oct 2012 09:25:12 +0200 (CEST)

> At the beginning of an SMTP transaction, all you have is a single IP.

Yes.

> the reputation of the IP address connecting to you.

Yes, and that reputation is derived from what subnet it's a subset of.

> You need an algorithm where you start from a single IP address and then 
> potentially "move up" until you get a meaningful result. That's more or 
> less what the B-tree algorithm suggested by John Levine some months ago 
> would offer: variable "depth" and "density" of data controlled by the 
> DNSxL operator optimized for the (on average) lowest number of lookups 
> needed.

Sounds good to me.

> data,third parties have to fall back to some default /64 etc.

Yes, which will be less than effective when customers are handed /48s and 
can send from anything within that /48.

>> What I feel needs to happen is that policy needs to put in place to 
>> RIRs (via ISPs) can present "what is a customer" on a network level, 
>> and then this information can be put into DNS somehow, and used for 
>> DNSBL.
>

RIPE already does (basically), as in "this /42 contains customers where 
each customer has a /56". I don't know about other RIRs.

---

From: "John Levine" <[email-masked]>
Date: 19 Oct 2012 22:41:31 -0000

>> this information can be put into DNS somehow, and used for DNSBL.

Yeah, I've been talking to people on and off about this for over a
year.  Even though providers can lie about their allocation
granularity, most won't, and the ones that lie would probably merit
total blocking anyway.

A customer allocation can be anywhere from a /56 to a /64, and I
expect we'll see hosting companies with a /64 per rack or per VM host,
so you'll have different customers within the same /64.

So, uh, anyone interested in doing simulations of how this sort of
stuff would work with various DNSBL designs?  I have access to IPv4
mailstream connection data we can use to calibrate it.

R's,
John

---

From: Dave Warren <[email-masked]>
Date: Fri, 19 Oct 2012 17:25:19 -0700

> granularity, most won't, and the ones that lie would probably merit

I'm less worried about those that lie outright than those that just 
don't care either by not bothering to specify a policy at all (unless it 
becomes mandatory somehow), or have more granularity than can be clearly 
specified in a single policy.

For example, their policy might be to allocate at the /64 level, but 
unless they also prohibit customers from obtaining more than one /64...

-- 
Dave Warren
http://www.hireahit.com/
http://ca.linkedin.com/in/davejwarren

---

From: Steve Atkins <[email-masked]>
Date: Fri, 19 Oct 2012 17:38:16 -0700

On Oct 19, 2012, at 5:25 PM, Dave Warren <[email-masked]> wrote:

> I'm less worried about those that lie outright than those that just don't care either by not bothering to specify a policy at all (unless it becomes mandatory somehow), or have more granularity than can be clearly specified in a single policy.
> 
> For example, their policy might be to allocate at the /64 level, but unless they also prohibit customers from obtaining more than one /64...

The ability for customers to obtain more than one IPv4 /32 hasn't been too complex an issue for blacklist operators to deal with. Any blacklist operator who's successfully running an IPv4 blacklist can surely come up with reasonable (or unreasonable, I won't judge…) policies for IPv6.

The only relevant difference between v4 and v6 DNS based blacklisting is that the ability to easily hop around *within* your /64 makes it possible (easy) to blow the cache of a traditional caching DNS resolver if you do naive "look up a record based on the IPv6 address".

That doesn't affect the viability of source address based blacklisting. It doesn't affect the viability of distributing that data as DNS zone files (they suck for both v4 and v6, but they're usable). And it doesn't affect the viability of using DNS as the communication channel between an MX and a local authoritative blacklist server.

But it does mean that anyone wanting a recursive resolver in their distribution path might have to refine the process a little - which is what I assume John is looking at. 

(I'm betting that "mask the bottom 64 bits before querying" would work just fine, but I don't think we have enough v6 space in use yet to say for sure.)

Cheers,
  Steve

---

From: Mikael Abrahamsson <[email-masked]>
Date: Sat, 20 Oct 2012 06:41:33 +0200 (CEST)

On Fri, 19 Oct 2012, Steve Atkins wrote:

> (I'm betting that "mask the bottom 64 bits before querying" would work 
> just fine, but I don't think we have enough v6 space in use yet to say 
> for sure.)

I agree. Although I believe some ISPs will put multiple customers in a 
single /64, this is mostly going to be dynamic customers anyway (multiple 
laptops on a wifi for instance), and those I would imagine are treated 
with the same policy so it doesn't really matter. Per /64 handling is a 
reasonable tradeoff between granularity and practicality in my mind.

---

From: Matthias Leisi <[email-masked]>
Date: Sat, 20 Oct 2012 08:55:37 +0200

On Sat, Oct 20, 2012 at 6:41 AM, Mikael Abrahamsson <[email-masked]> wrote:

> laptops on a wifi for instance), and those I would imagine are treated with
> the same policy so it doesn't really matter. Per /64 handling is a

And cheap VPS, where the "upgrade" from /128 to /64 is used as a
differentiator (just stumbled upon such a case,
http://www.kimsufi.com/fr/vks/).

-- Matthias

---

From: "Peter J. Holzer" <[email-masked]>
Date: Sat, 20 Oct 2012 09:30:31 +0200

On 2012-10-19 17:38:16 -0700, Steve Atkins wrote:
> > don't care either by not bothering to specify a policy at all
> > (unless it becomes mandatory somehow), or have more granularity than
> > can be clearly specified in a single policy.
> > 
> The ability for customers to obtain more than one IPv4 /32 hasn't been
> too complex an issue for blacklist operators to deal with. Any
> blacklist operator who's successfully running an IPv4 blacklist can
> surely come up with reasonable (or unreasonable, I won't judge…)
> policies for IPv6.
> 
> The only relevant difference between v4 and v6 DNS based blacklisting
> is that the ability to easily hop around *within* your /64 makes it
> possible (easy) to blow the cache of a traditional caching DNS
> resolver if you do naive "look up a record based on the IPv6 address".

A simple (maybe naive) idea to reduce cache poisoning would be to do
some kind of greylisting before doing the lookup: If you haven't seen
the address before, simply return a temporary error. If you have seen
it (within some time window), do the lookup. 

Is there a reason why a legitimate MTA (talking to MXs, not submission
servers) would want to hop around in its net?

 * IPv6 privacy extensions might be turned on by default and the admin
   might not bother to turn them off, but I think they are too slow to
   prevent delivery (although it will slow down most mails)
 * An admin of an MTA serving many customers might want to use a
   different IP address for each customer. But from the outside that 
   would just look like one MTA per customer, not a single MTA hopping
   around

> That doesn't affect the viability of source address based
> blacklisting. It doesn't affect the viability of distributing that
> data as DNS zone files (they suck for both v4 and v6, but they're
> usable).

There are many more IPv6 addresses and even /64 nets than IPv4
addresses. It's theoretically possible to ship around a zone file with
/all/ IPv4 addresses (that would be less than 100GB). This isn't remotely 
possible for IPv6 space. It might be possible for allocated /64 nets
depending on how they are allocated (if every cell phone gets a /48 ...).
If you go down to the /128 level, a spammer doing the hopping maneuvre
could blow up your zone file beyond any reasonable limit.

> And it doesn't affect the viability of using DNS as the
> communication channel between an MX and a local authoritative
> blacklist server.
[...]

Right.

	hp

-- 
   _  | Peter J. Holzer    | Der eigene Verstand bleibt gefühlt messer-
|_|_) | Sysadmin WSR       | scharf. Aber die restliche Welt blickt's
| |   | [email-masked]         | immer weniger.
__/   | http://www.hjp.at/ |   -- Matthias Kohrs in desd

---

From: Bart Schaefer <[email-masked]>
Date: Sat, 20 Oct 2012 07:25:04 -0700

On Oct 20,  9:30am, Peter J. Holzer wrote:
}
} Is there a reason why a legitimate MTA (talking to MXs, not submission
} servers) would want to hop around in its net?

A legitimate MTA could still be running in a dynamically-assigned space.
In this case it might hop all over the space but probably wouldn't hop
very frequently.

A single MTA host might have multiple NICs each with its own IP, and not
always choose the same interface for the same MX on retry.  Here it might
hop quite a lot, but among a limited number of choices.

---

From: "John Levine" <[email-masked]>
Date: 20 Oct 2012 21:40:48 -0000

>The only relevant difference between v4 and v6 DNS based blacklisting is
>that the ability to easily hop around *within* your /64 makes it
>possible (easy) to blow the cache of a traditional caching DNS resolver
>if you do naive "look up a record based on the IPv6 address".

That's always been our assumption, but there is at this point precious
little evidence that MTAs that query DNSBL through caches (as opposed
to those who have a local rsync mirror) have a cache hit rate much
greater than zero.  If in fact they don't, the same design, perhaps
with a little TTL tuning to give clients a hint about which ones are
worth caching, could work no worse than they do now for similar mail
volumes.

That's why I want to do the cache simulations, to figure out what's
going on now.  It shouldn't be terribly hard, and we have people
offering data.  Want to help?

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 20 Oct 2012 21:42:57 -0000

>} servers) would want to hop around in its net?

Probably not, although I'm waiting for ESPs to figure out that if they
send every message from a different IP, it'll be much easier to
process bounces and complaints since all they'll need is the IP to
figure out what the list and address was.

Bad guys could use it to listwash, of course, but it's not totally
ridiculous.

R's,
John

---

From: "Peter J. Holzer" <[email-masked]>
Date: Sun, 21 Oct 2012 23:20:40 +0200

On 2012-10-20 21:42:57 -0000, John Levine wrote:
> send every message from a different IP,

I thought of that but I wouldn't be surprised to overflow the router's
ARP table even with our moderately sized mailing-lists (a few thousand
subscribers at most). That's why I only mentioned "one address per
customer", not "one address per message" as a likely tactic.

> it'll be much easier to process bounces and complaints since all
> they'll need is the IP to figure out what the list and address was.

Is it? For mailing-lists, I think VERP is simpler and more robust. The
IP address is buried somewhere in the Received headers of the bounced
message, so you have to parse those. For complaints to an ISP about a
customer that might indeed be useful. Depends on what information is
included in the complaint. If it contains the complete header of the
message there is probably other identifying information. It it doesn't,
chances are that the IP address isn't included, either.

> Bad guys could use it to listwash, of course, but it's not totally

There are other ways to listwash. I'm more worried that the bad guys are
using rapidly changing IP addresses to escape or overflow BLs.

	hp

---

From: "Peter J. Holzer" <[email-masked]>
Date: Sun, 21 Oct 2012 23:37:08 +0200

> In this case it might hop all over the space but probably wouldn't hop

By "dynamically-assigned space" do you mean a dynamically assigned
address within a /64 (either by DHCP or by privacy extensions)? If so, I
already mentioned that and yes, I think it doesn't change fast enough to
make greylisting infeasible (but frequently enough to make it annoying).

If you mean that an ISP is assigning a different /64 to the same
customer periodically (some privacy evangelists are demanding that this
should be the default), then this would probably be done even less
frequently, and this would most likely be treated the same as
dynamically assigned space today (i.e. very likely to be a zombie, not a
legitimate MTA).

> hop quite a lot, but among a limited number of choices.

An IP stack might also choose IP addresses at random or in a round robin
fashion if the interface has several. That could be a problem.

	hp

---

From: "John Levine" <[email-masked]>
Date: 21 Oct 2012 22:44:06 -0000

>ARP table

Well, yes, if you configure stuff in a naive way.  I'd route an entire
/64 to the mail server, and configure the server as its own router
with all of those IP addresses internally forwarded to a single IP
that talks to the outside world.  If you wanted to do IP hopping,
it wouldn't be hard to do.

>Is it? For mailing-lists, I think VERP is simpler and more robust.

Depends what your intentions are.  If you're trying to do listwashing,
you may wall see DNSBL listings rather than bounces.  I like VERP just
fine and my lists use it, but I do get back FBL reports that are
munged to the point where I can't tell who complained.  But they
rarely munge the IP.

R's,
John

---

From: Tim Chown <[email-masked]>
Date: Mon, 22 Oct 2012 11:26:46 +0100

On 20 Oct 2012, at 05:41, Mikael Abrahamsson <[email-masked]> wrote:

> I agree. Although I believe some ISPs will put multiple customers in a single /64, this is mostly going to be dynamic customers anyway (multiple laptops on a wifi for instance), and those I would imagine are treated with the same policy so it doesn't really matter. Per /64 handling is a reasonable tradeoff between granularity and practicality in my mind.

A /64 basis seems to be the most sensible starting point, from which one could 'suck it and see'.

In client scenarios where multiple users share a /64, e.g. in a wifi hotspot, then they are probably sharing one public IPv4 address with IPv4 NAT now.  

In data centre scenarios, what do we expect there? It's not clear from draft-lopez-v6ops-dc-ipv6-03.  It may depend on whether the hosting is physical or virtual? The allocated prefix is likely to be quite stable.

In home networks, the current trend in Europe seems to be /60 to /56, though some offer /64 or /48.  In some cases the equipment can only handle /64 now, but expect that to shift to /56 or /60 later.  The work on draft-ietf-homenet-arch-05 expects the ISP to offer multiple /64's.  It seems many ISPs will vary the prefix offered over time much as with dynamic IPv4 addresses today.  In some countries, that's apparently a legal requirement.

In enterprises, /48, and more likely to be stable.

Our evidence from incoming spam is that most of it comes via dual-stack list servers.  There's very little evidence of spam coming from autoconfigured addresses that would indicate client senders.

Tim