Subject: 6to4 relay and Teredo relay load - Bittorrent

From: Nathan Ward <[email-masked]>
Date: Sat, 13 Oct 2007 13:31:10 +1300

Hi all,

Are any people operating 6to4 or Teredo relays publishing graphs of  
packetrate and bitrate through their relays?

I've just observed my Bittorrent client (Azureus) doing what I assume  
is DHT (Distributed Hash Table) stuff over IPv6.

The concern I have is that it's doing stuff from my 6to4 address to  
'native' v6 addresses, and Teredo addresses - which would produce a  
higher load on the public relays for those protocols (probably).

So, anyone out there who can share data with me? :-)

--
Nathan Ward

ps. I'd actually planned on getting in touch with the Azureus guys  
and trying to get IPv6 integrated in to their client - but make it  
behave in such a way that it was nice to the relays, at least to  
start with. I was slack.

---

From: Jeroen Massar <[email-masked]>
Date: Sat, 13 Oct 2007 11:35:06 +0200

[cc'd to [email-masked] which contains network operators and
not the folks defining how to operate it]

> packetrate and bitrate through their relays?

I can only assume that it is less than 10mbit in most cases, this
looking at AMS-IX traffic graphs[1] and the ones from SixXS[2], the
latter being almost quite a large bit of the AMS-IX traffic graphs (even
has the same forms, which is quite funny). As AMS-IX is only doing
50mbit on peek time, when Teredo/6to4 traffic is more than 10mbit it
would be more amazing than Mars[3].

Of course this is a bit of guess work and Teredo/6to4 traffic might
partially avoid IX's or show up as IPv4 there.

It would be fun to get an analysis of a heavy box though. But I can only
assume that the usages are primarily: NNTP, bit of HTTP, SMTP and IRC.

> I've just observed my Bittorrent client (Azureus) doing what I assume is
> DHT (Distributed Hash Table) stuff over IPv6.
> 

If you don't want or can't handle the traffic then either:
 - limit how far your 6to4 relay announces itself in BGP
 - don't run it

and other similar options.

This is btw why I am not a big proponent of 6to4: as a network operator
you don't have much control over the path taken for packets by 6to4
hosts, nor who uses the relay in your network.

> ps. I'd actually planned on getting in touch with the Azureus guys and
> trying to get IPv6 integrated in to their client - but make it behave in
> such a way that it was nice to the relays, at least to start with. I was
> slack.

Be happy that they already have IPv6 implemented, BT might actually be
one of the few things that might generate lots of traffic on IPv6.
Current IPv6 traffic levels are simply pathetic.

For folks wanting to know which BitTorrent clients work over IPv6,
please check: http://www.sixxs.net/tools/tracker/clients/
That system also has an IPv6-only tracker service where people can
upload their files upto and it will be properly seeded for other folks
to download.

Greets,
 Jeroen

[1] http://www.ams-ix.net/technical/stats/sflow_stats_ipv6.html
[2] http://www.sixxs.net/misc/traffic/
[3] http://nl.youtube.com/watch?v=6-OnKakA8TY :)

---

From: james woodyatt <[email-masked]>
Date: Sat, 13 Oct 2007 09:07:03 -0700

> one of the few things that might generate lots of traffic on IPv6.

Highly unlikely.  Ubiquitous IPv6 stateful packet filters that behave  
the same way as the filters in IPv4/NAT gateways will give BitTorrent  
users the same hassles on IPv6 as they currently endure on IPv4/NAT.   
The incentive for BitTorrent users to migrate to IPv4 simply isn't  
there unless they're willing to turn off those filters.

So far, there appears to be no let up in the demand for these filters  
on IPv6.  It's possible that a long-term solution to this problem  
might emerge in five or ten years, but I wouldn't expect much before  
then.

--
james woodyatt <[email-masked]>
member of technical staff, communications engineering

---

From: Rémi Denis-Courmont <[email-masked]>
Date: Sat, 13 Oct 2007 20:11:05 +0300

> there unless they're willing to turn off those filters.

Precisely not if they use 6to4 and Teredo à la Windows way of doing unmanaged 
IPv6 transition.

-- 
Rémi Denis-Courmont
http://www.remlab.net/

---

From: Nathan Ward <[email-masked]>
Date: Sun, 14 Oct 2007 12:24:44 +1300

On 14/10/2007, at 5:07 AM, james woodyatt wrote:

> Highly unlikely.  Ubiquitous IPv6 stateful packet filters that  
> behave the same way as the filters in IPv4/NAT gateways will give  
> BitTorrent users the same hassles on IPv6 as they currently endure  
> on IPv4/NAT.  The incentive for BitTorrent users to migrate to IPv4  
> simply isn't there unless they're willing to turn off those filters.
>
> So far, there appears to be no let up in the demand for these  
> filters on IPv6.  It's possible that a long-term solution to this  
> problem might emerge in five or ten years, but I wouldn't expect  
> much before then.

Of the hosts that attempted to connect to me:
- Over 50% had 135/TCP open.
- Several had 22/TCP open, and everything else closed (returning RST).
- One had 80/TCP open, and everything else simply dropping packets.

That 135/TCP is open is certainly surprising - these were all hosts  
using the Windows-like single-host-on-6to4 addressing, ie.  
2002:V4V4:V4V4::V4V4:V4V4.

--
Nathan Ward

---

From: Nathan Ward <[email-masked]>
Date: Sun, 14 Oct 2007 17:52:17 +1300

On 14/10/2007, at 12:24 PM, Nathan Ward wrote:

>> on IPv4/NAT.  The incentive for BitTorrent users to migrate to  
>> IPv4 simply isn't there unless they're willing to turn off those  
>> filters.
>>

I did some more digging. Here's some interesting data:
- 1 hour of collecting packets directed to my BT client's DHT port  
(it uses UDP).
- 3134 unique hosts talked to me.
- 1535 IPv4
- 1599 IPv6 (!!)

IPv6 hosts:
- 20 were non-6to4 (ie outside 2002::/16)
- zero were Teredo (ie 2001::/32)

IPv6 6to4 (2002::/16) hosts:
- 1288 were "Windows-style" 6to4 addresses - as mentioned above.
- 19 were using a non-zero "subnet" part (ie, bits 49-64 inclusive),  
more detail below.

Non-windows style addresses (291 hosts):
- 277 were using EUI-64, and all had the locally administered bit set.
- 14 were not using EUI-64, 9 of these had a non-zero subnet part.  
Only one had :: in the address, bits 49-128 were 1::1.
- No MAC-48.

6to4 "subnet" part (only 19 hosts):
- Only one had 0x0001 in the subnet part, and zero had 0x000a or  
greater.

The bit I find the most astonishing, is the ratio of v4:v6 hosts. My  
understanding of BT's DHT is that it is used to find peers to  
exchange data with - this means that if DHT is effective, there's  
likely to be a huge amount of BT data between DHT peers going over  
6to4. Luckily, only a very small amount (0.63%) is likely to go via a  
6to4 relay.

Also interesting is where people are not using EUI-64. I guess this  
is using the privacy extensions stuff.

I noted that of the hosts I attempted to send DHT data to, the ones  
that didn't reply were about 20% IPv6 and 80% IPv4. As I look higher  
in the number of reply packets, the hosts that replied were about 95%  
IPv6 and 5% IPv4. That indicates to me that IPv6 is working around  
the IPv4 NAT problem better than the available IPv4 solutions for  
this application.
Of the traffic coming to my IP addresses, 62% was IPv6 and 38% was IPv4.

I'll see if I can get "replied" percentages for IPv4 and IPv6, where  
I sent data to a host and got a reply vs. didn't get a reply. I'm not  
sure if that's relevant though, as this is UDP and might not have any  
kind of response required at a higher level.
If there is a response required, I'll see if I can find some hosts  
who I have talked to over both IPv4 and IPv6, and see if there's any  
notable lack of replies when using either protocol.

--
Nathan Ward

---

From: Nathan Ward <[email-masked]>
Date: Mon, 15 Oct 2007 13:20:12 +1300

On 14/10/2007, at 5:52 PM, Nathan Ward wrote:
> <numbers>

Exciting times.

 From the Azureus changelog for v3.0.2.0, August 20 2007:
- Core | IPv6 support - dht, version server, tracker and improved  
transport

This corresponds with at least one 6to4 relay's increase in traffic.  
(only up about 1kbit/s, but it's about 2-3 times what it was before.

I believe that this particular relay is in front of a large number of  
broadband and dial customers, but there appears to be very little  
'intentional' v6 traffic that traverses it, as it's mainly average  
home/business users.

--
Nathan Ward

---

From: Kevin Loch <[email-masked]>
Date: Mon, 15 Oct 2007 00:34:09 -0400

Nathan Ward wrote:
> - Core | IPv6 support - dht, version server, tracker and improved transport
> 

I also saw a slight increasae in traffic on 2007/08/20.  It's more
noticeable in the pps graph than the bits/s.  This relay
(AS33437/AS29748) currently averages about 1mbit/s.

- Kevin

---

From: Nathan Ward <[email-masked]>
Date: Fri, 19 Oct 2007 20:04:27 +1300

On 15/10/2007, at 5:34 PM, Kevin Loch wrote:
>> This corresponds with at least one 6to4 relay's increase in  
>> traffic. (only up about 1kbit/s, but it's about 2-3 times what it  
>> was before.
>

OK.

As you'll note from my stats from my packet traces, very few would  
have come through a relay, as most are 6to4, so it's difficult to  
tell how much this has increased 6to4 traffic between end sites.

Though, I'm going to have a look at the code and attempt to figure  
out if it has some way of optimising things to prefer 6to4 if it is  
on 6to4 itself, etc. (I am on 6to4 myself, so if there is a  
preference like that, it would explain the higher amount of 6to4 vs.  
other).

--
Nathan Ward