Subject: Implications of v6 on application level rate limiting...

From: "Alexander Mayrhofer" <[email-masked]>
Date: Fri, 5 Feb 2010 15:30:58 +0100

Hi,

I'm trying to find documents about the implication of v6 on rate
limiting on application level. I've gone through the list of v6ops
documents, but haven't found any reference to or guidance about the
following issue:

I'm working for the ".at" ccTLD registry, and we're currently
investigating the implications of adding v6 transport to our WHOIS
servers. For data protection & load management reasons, those WHOIS
servers are currently configured to apply rate limiting on the
application level on a per-IPv4-address basis, for example "5 queries
per hour, 100 queries per day" (etc). This works quite well on IPv4,
since acquiring a new IPv4 address is not trivial in most scenarios.

With the introduction of IPv6, the "per IP" strategy obvioulsy doesn't
work anymore like this, because any host with a /64 can essentially
generate a new IP address for each request.

A simple approach would be to aggregate requests by prefix (/64 or /56
or even /48?), and use that prefix instead of the full IP adress. This
problem is not specific to our WHOIS use case, but will show up in SMTP
rate limiting, ssh blacklisting applications, SIP registration servers,
etc..

Since this problem is therefore pretty generic, my question is whether
the v6ops working group has created considerations for such cases. If
not: Would the v6ops be chartered to do such work, and is interest in
this problem big enough to get something started?

I beg to pardon if this issue has been discussed/documented before, and
i have missed it. I'd appreciate being pointed at the work in that case
.. 

Comments are appreciated.

Thanks,

Alex

---

From: Jeroen Massar <[email-masked]>
Date: Fri, 05 Feb 2010 09:57:01 -0600

Alexander Mayrhofer wrote:
[..]
> rate limiting, ssh blacklisting applications, SIP registration servers,

Indeed, that is the most simple and obvious approach: per 'level' eg
chunked something in order of /64, /48, /40, /36, /32 if X
hosts/upper-levels in that level do something bad you aggregate to the
next level.
X could vary per level of course.

Very crude, but very effective, as at a /32 you will have blocked the
full ISP if they are 'bad' as seen from your policy point of view.

Greets,
 Jeroen

---

From: Marco Hogewoning <[email-masked]>
Date: Fri, 5 Feb 2010 17:44:29 +0100

> X could vary per level of course.

And please please please make this aggregation policy case based or never aggregated above /64. There are a lot of ISP's out there who have different productsm  with very different address plans which are all in the same /32 and can be quite close together, with a default policy of /56 for broadband residential, /48 for business and /64 for mobile.

So what seem to be a good and decent aggregation for DSL can all of a sudden create huge problems as you unintentionally block or ratelimit 65.000 mobile users from that same ISP.

> full ISP if they are 'bad' as seen from your policy point of view.

Not connecting to the internet at all might be a better choice for those people :)

Groet,

MarcoH

---

From: Adrian Kennard <[email-masked]>
Date: Sat, 06 Feb 2010 08:34:36 +0000

> So what seem to be a good and decent aggregation for DSL can all of a sudden create huge problems as you unintentionally block or ratelimit 65.000 mobile users from that same ISP.

I fear the same question will arise for people running spam black lists.

The space is so large that the allocation policies can vary massively 
from one /32 to the next. At one extreme an LIR with a /32 need not even 
be a provider and may allocate blocks to separate providers so 
aggregating at the /32 would be silly as the LIR has no control over 
then end users at all. In other cases millions of separate users may 
have a single device IP allocated on a /127 PPP link.

If it is any help, as an ISP that has been doing IPv6 on DSL for 7 
years, we allocate a /48 to each distinct customer regardless of how 
many lines they have. We give them control over what blocks (down to 
/64) are allocated to each line or lines as they need.

Of course you could whois the requester IPv6 address to see the block 
size allocated to the requesting end user. But the whois server 
providing that information might rate limit you !!!

---

From: Marco Hogewoning <[email-masked]>
Date: Sat, 6 Feb 2010 13:18:58 +0100

> The space is so large that the allocation policies can vary massively from one /32 to the next. At one extreme an LIR with a /32 need not even be a provider and may allocate blocks to separate providers so aggregating at the /32 would be silly as the LIR has no control over then end users at all. In other cases millions of separate users may have a single device IP allocated on a /127 PPP link.
> 
> If it is any help, as an ISP that has been doing IPv6 on DSL for 7 years, we allocate a /48 to each distinct customer regardless of how many lines they have. We give them control over what blocks (down to /64) are allocated to each line or lines as they need.
> 
> Of course you could whois the requester IPv6 address to see the block size allocated to the requesting end user. But the whois server providing that information might rate limit you !!!

I made a suggestion to one of the ripe lists on how we might fix that a couple of months ago, details are here:

http://www.ripe.net/ripe/maillists/archives/ipv6-wg/2009/msg00142.html

Now this was just an idea I have and either people don't find it important enough or think it's stupid, as I haven't seen much response on it I fear the first is true. Or maybe I just missed it and everybody running blacklists found a way to safely predict which blocks can be easily aggregated or which don't.

I had a discussion with some people about this and one of the suggestions made was that maybe the IETF would be a more suitable place to try and fix this as it has global reach and you don't have to go over the exercise 5 times for each individual registry. Somebody even suggested we might fix this by using DNS as a publication method.

I think the base requirement is to find a way in which operators can easily publish some hints on where aggregation boundaries are, possibly together with some contact information on who to notify about issues.

This system:

- has to be easily filled and updated by operators
- has to be even easier to use for those people trying to block/limit IPv6 address blocks
- well documented API for automatic retrieval
- preferably uses a single method or entrypoint to get data on all RIRs
- either has a distributed architecture or caching mechanism to hande the load

Once this is in place it can benefit both sides, you don't unintentionally block a whole lot of users and at the same time prevent virusses/trojans/zombies from address hopping to try and circumvent blocklists. I can also see some gain from a operational perspective, as the whole customer is shutdown in one go instead of only a handfull of his million billion addresses, it's much easier to identify the problem.

MarcoH

---

From: Roque Gagliano <[email-masked]>
Date: Sun, 7 Feb 2010 01:13:53 +0100

hi Jeroen,

> chunked something in order of /64, /48, /40, /36, /32 if X

The problem that I see using any length shorter than 48 is that RIR's min allocations are /48s. This means that if you block a /32, you may be blocking 65000 end-users assignments. So, if you want to use any length shorter than 48, you would need to follow RIR's allocation sizes in their allocated address space, and that changes in time with new policies.

r.

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 07 Feb 2010 13:55:52 +1300

> The problem that I see using any length shorter than 48 is that RIR's min allocations are /48s. This means that if you block a /32, you may be blocking 65000 end-users assignments. So, if you want…

Since there are ISPs assigning /64 to some customers, a policy
that blindly blocks /48 or /56 will be problematic (lawyer bait,
even).

     Brian

---

From: SM <[email-masked]>
Date: Sun, 07 Feb 2010 06:56:52 -0800

Hi Alex,
>rate limiting, ssh blacklisting applications, SIP registration servers,

There is a DNSBL listing the /64 if there are reports for five hosts 
within that /64.   SMTP rate limiting is different from WHOIS as it 
is MTA to MTA communication.  WHOIS, as several other application 
protocols that you mentioned, is user agent to server 
communication.  If your limit is 100 queries per day per /64, that 
limit would be reached if the queries are spread over several clients 
in that /64.  In practice, it may be viewed as acceptable to rate 
limit per /64.

Regards,
-sm

---

From: Pekka Savola <[email-masked]>
Date: Mon, 8 Feb 2010 12:46:07 +0200 (EET)

> i have missed it. I'd appreciate being pointed at the work in that case

Some 6-7 years ago, I recall talking about this at an IETF with folks 
from RIPE NCC which had similar issues with the RIPE DB. 
Unfortunately I can't find which document this was about; it may have 
been what became RFC4942, but the only text there that has some 
resemblance to this problem space is in S 2.1.7.  Sorry for being 
vague.  I don't think this helps much, but that's my 5c.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings