Subject: Re: [v6ops] my recommendation re: 6to4

From: Iljitsch van Beijnum <[email-masked]>
Date: Sat, 1 Nov 2014 10:06:20 +0100

On 01 Nov 2014, at 0:07, Templin, Fred L <[email-masked]> wrote:

> Just a couple of quick comments:

Famous last words... Here are mine:

I haven't followed the previous discussions, but what problem are we trying to solve, exactly?

Yes, 6to4 has tons of issues. All the same, there are still people using it out there today, and I would hazard to guess that they don't have a better alternative. So why make life worse for them by going out of our way to kill 6to4?

Implementations already prefer IPv4 over 6to4, so I don't think having 6to4 enabled causes too many problems these days anyway.

>> 4.  However the above actions are labeled, they make it all the more obvious that the Internet community lacks an obvious, recommended, and generally-applicable IPv6-over-IPv4 tunneling mechanism that can be implemented in a host or router

> Maybe, but it does  not necessarily need to be ip-proto-41 based.

I'll go one step further and say it MUST NOT be protocol 41 based.

> And, it need not be specific to IPv6-over-IPv4 – it would be much better to have a single mechanism that can support tunneling of IPvX-over-IPvY for any values of X and Y.

Not so sure. If it's strictly IPv6-over-IPv4 there are many optimizations and simplifications that are possible.

However, if we want to go down this path (and we should have done that 10 years ago, it could very well be too late now) I strongly suggest we do it right and start with requirements. Here are a few:

- if at all possible, autoconfigure everything, if not, configuration should be limited to something no more complex than a username/password
- must be reliable and predictable, so if A - B works and B - C works, A - C should also work 99.9% of the time
- explicit reachability check, so traffic doesn't continue to be sent into a black hole
- work through NAT
- work through CGN
- automatically adapts to changing addresses
- low overhead in gateways (preferably completely stateless)
- can be terminated on a home gateway or on a host
- doesn't require ISP cooperation
- traffic flows are reasonably optimized (i.e., if ISP offers gateway service, by default, that gateway is used)

As I read this, I'm pretty sure that some of the existing tunnel broker stuff that works over UDP today already does pretty much all of this as far as the actual tunneling goes. What would be needed is more streamlined signup and setup processes.

Perhaps it could look like this: as a user, you either automatically get an account using your existing email / cloud service credentials, or you sign up for a separate account, like [email-masked]. You enter your username and password into the home gateways and hosts that you want to have tunneled IPv6 connectivity.

The router or host, when it notices that there is no native IPv6, connects to _tunserv._tcp.<domain> or something like that and authenticates. The setup server then does a lookup of the source IPv4 address and redirects to a gateway serving that address range. Ideally, that would be the ISP providing service for those addresses. Or a public gateway that has good connectivity to those addresses. (There would be communication between the gateway and setup server operators to avoid lame delegations. And perhaps the setup server returns multiple delegations so if one gateway doesn't work the client tries the next one.)

The router or host then talks to that gateway to obtain a prefix and other configuration info and sets up the tunnel. The host or home gateway is then responsible for keeping NAT mappings open and reconnecting when connectivity goes away for some reason.

I guess it would be possible for devices to log in using their MAC address by default. Not sure how much authentication we really need for this. An account would still be better because that way any errors could be communicated through email. And logging in using a known service means the user's activities aren't exposed to additional parties.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Sat, 1 Nov 2014 10:14:43 +0100

On 01 Nov 2014, at 10:06, Iljitsch van Beijnum <[email-masked]> wrote:

> Perhaps it could look like this: as a user, you either automatically get an account using your existing email / cloud service credentials

[...]

Another thing: we could set aside a range of IPv6 addresses to be used by "tunnels of last resort". These addresses would get a lower priority than IPv4 addresses in the policy table, so if a tunnel broker / gateway hands out these addresses it can provide IPv6 connectivity towards IPv6-only destinations, but dual stack destinations are handled over IPv4, so the load on the gateway would be much less than in the situation where regular IPv6 addresses with a higher priority than IPv4 addresses are used.

3ffe::/16 perhaps?  </evil>

---

From: "Templin, Fred L" <[email-masked]>
Date: Sat, 1 Nov 2014 15:21:51 +0000

Hi Iljitsch,

> -----Original Message-----
> From: Iljitsch van Beijnum [mailto:[email-masked]]
> Sent: Saturday, November 01, 2014 2:06 AM
> To: Templin, Fred L
> Cc: Keith Moore; [email-masked] WG
> Subject: Re: [v6ops] my recommendation re: 6to4
> 
> Yes, 6to4 has tons of issues. All the same, there are still people using it out there today, and I would hazard to guess that they don't
> have a better alternative. So why make life worse for them by going out of our way to kill 6to4?
> 
> >> 4.  However the above actions are labeled, they make it all the more obvious that the Internet community lacks an obvious,
> recommended, and generally-applicable IPv6-over-IPv4 tunneling mechanism that can be implemented in a host or router
> 

OK.

> > And, it need not be specific to IPv6-over-IPv4 – it would be much better to have a single mechanism that can support tunneling of
> IPvX-over-IPvY for any values of X and Y.
> 

With respect to AERO at least; the only advantage I see is that encapsulation over IPv4
instead of IPv6 saves 20 bytes per packet. I guess also that dealing with "default" is
simpler than for IPv6/foo/IPv6 encapsulation.

> However, if we want to go down this path (and we should have done that 10 years ago, it could very well be too late now) I strongly

Too late? If we can be 20yrs into this and still not have ubiquitous IPv6 deployment
then we are clearly not too late. IPv6 is a very old protocol - ancient really - we just
need to learn to deal with it in new ways.

> suggest we do it right and start with requirements. Here are a few:
> 

Or a signed certificate?

> - must be reliable and predictable, so if A - B works and B - C works, A - C should also work 99.9% of the time

Transitive reachability can never be assumed, and needs to be tested.  I can't
give you a % that would hold true for every deployment scenario.

> - explicit reachability check, so traffic doesn't continue to be sent into a black hole

IPv6 ND Neighbor Unreachability Detection (NUD)

> - work through CGN

No problem there.

> - automatically adapts to changing addresses

You mean for mobility support? Sure.

> - low overhead in gateways (preferably completely stateless)

Gateways that terminate a tunnel, or gateways that forward tunneled packet?

> - can be terminated on a home gateway or on a host

OK.

> - doesn't require ISP cooperation

OK.

> - traffic flows are reasonably optimized (i.e., if ISP offers gateway service, by default, that gateway is used)

Route optimization, yes.

> As I read this, I'm pretty sure that some of the existing tunnel broker stuff that works over UDP today already does pretty much all of
> this as far as the actual tunneling goes. What would be needed is more streamlined signup and setup processes.

DHCPv6 PD?

> Perhaps it could look like this: as a user, you either automatically get an account using your existing email / cloud service credentials, or
> you sign up for a separate account, like [email-masked]. You enter your username and password into the home
> gateways and hosts that you want to have tunneled IPv6 connectivity.

The devices need to be enrolled in the service, sure.

> The router or host, when it notices that there is no native IPv6, connects to _tunserv._tcp.<domain> or something like that and
> authenticates. The setup server then does a lookup of the source IPv4 address and redirects to a gateway serving that address range.
> Ideally, that would be the ISP providing service for those addresses. Or a public gateway that has good connectivity to those
> addresses. (There would be communication between the gateway and setup server operators to avoid lame delegations. And
> perhaps the setup server returns multiple delegations so if one gateway doesn't work the client tries the next one.)

The Client should be able to check for a different Server if one or more other
Servers are not working, yes.

> The router or host then talks to that gateway to obtain a prefix and other configuration info and sets up the tunnel. The host or home
> gateway is then responsible for keeping NAT mappings open and reconnecting when connectivity goes away for some reason.

Huh? If the connection dies, the NAT mappings will be re-established when the connection
is re-established.

> I guess it would be possible for devices to log in using their MAC address by default. Not sure how much authentication we really need
> for this. An account would still be better because that way any errors could be communicated through email. And logging in using a
> known service means the user's activities aren't exposed to additional parties.

DHCPv6 DUID does not need to be based on MAC address.

Thanks - Fred
[email-masked]

---

From: Owen DeLong <[email-masked]>
Date: Sat, 1 Nov 2014 18:45:43 -0700

> I'll go one step further and say it MUST NOT be protocol 41 based.

Why not? What's wrong with explicitly configured 6in4 tunnels? I'll grant they don't buy much vs. protocol 45 IPv6 in GRE/IPv4 tunnels, but there's a little less overhead and the mechanism is already defined, working, and in fairly wide implementation in various systems, routers, etc.

> Not so sure. If it's strictly IPv6-over-IPv4 there are many optimizations and simplifications that are possible.

That seems to argue for preserving 6in4 over protocol 41.

> - if at all possible, autoconfigure everything, if not, configuration should be limited to something no more complex than a username/password

APIs can accomplish this for 6in4, actually. It's not there yet, but there's no reason this couldn't be built on existing systems.
http://tunnelbroker.net already does this for the "server-side" of the equation. Username/Password and basically point-click-tunnel.

> - must be reliable and predictable, so if A - B works and B - C works, A - C should also work 99.9% of the time

I would argue that is by and large the case with HE's protocol 41 tunnelbroker. Can you point to reasons you believe otherwise?

> - explicit reachability check, so traffic doesn't continue to be sent into a black hole

This would be a lovely addition, but I think it can be added either to the existing protocol 41 tunnels or on top of them.

For example, to some extent, my BGP over 6in4 tunnels have this feature for some interpretations of the request.

> - work through CGN

Meh... There are tunnel mechanisms that achieve this, but they create unnecessary pain in other areas where these are not a factor. OTOH, I'm not sure there needs to be "one true tunneling mechanism". Requiring the additional overhead of UDP in all cases and STUN in general doesn't feel like a win in the cases where NAT traversal is not required.

> - automatically adapts to changing addresses

Harder problem to solve. I could, however, do this today with some development work on an HE tunnel.

> - low overhead in gateways (preferably completely stateless)

6in4 already meets this requirement.

> - can be terminated on a home gateway or on a host

6in4 already meets this requirement.

> - doesn't require ISP cooperation

Since all packet forwarding requires some level of ISP cooperation, I think you need to be a little more clear about your meaning here.
If you think this eliminates 6in4 because some ISPs are obnoxious enough to block protocol 41, then I would suggest there is a reasonable likelihood that said ISPs will block whatever they have to in order to block IPv6 tunneling for the same reasons they blocked protocol 41.

If you mean something else, then please explicate.

> - traffic flows are reasonably optimized (i.e., if ISP offers gateway service, by default, that gateway is used)

This requires some form of service discovery mechanism that can somehow be aware of what ISP you are using or otherwise has some mechanism for automatically interfacing with said ISPs provisioning. In the tradition of rough consensus and running code, it would be great if you put forward your vision for how this would work.

> As I read this, I'm pretty sure that some of the existing tunnel broker stuff that works over UDP today already does pretty much all of this as far as the actual tunneling goes. What would be neede…

I think that could be achieved as an API on top of what exists. However, see my above comments as to why I don't think that should be the only available solution and I definitely think that deprecation of configured 6in4 tunnels would be very premature and somewhat harmful in general.

> Perhaps it could look like this: as a user, you either automatically get an account using your existing email / cloud service credentials, or you sign up for a separate account, like [email-masked]…

It's not UDP based, but FWIW there is an API to the HE tunnelbroker that was designed to allow home gateway implementers to do essentially just that through the web API on their gateways.

> The router or host, when it notices that there is no native IPv6, connects to _tunserv._tcp.<domain> or something like that and authenticates. The setup server then does a lookup of the source IPv4…

This assumes a tremendous amount of cooperation and coordination not in evidence. Who runs this central "setup server" service? Why? How is it paid for?

> I guess it would be possible for devices to log in using their MAC address by default. Not sure how much authentication we really need for this. An account would still be better because that way an…

There's also the issue that MAC addresses are only guaranteed to be segment unique. There are many cases where they are _NOT_ globally unique.

Owen

---

From: "Templin, Fred L" <[email-masked]>
Date: Sun, 2 Nov 2014 05:48:54 +0000

FWIW, AERO works fine over ip-proto-41 when UDP is not needed. It's just that there
are lots of places UDP can go that ip-proto-41 can't - and that is without even taking
NATs into account. (UDP can also be easier to work with in application-layer
implementations, but that is a different subject.)

Also, ip-proto-41 does have path MTU challenges. We all know that PMTUD is
unreliable, which means the tunnel ingress has to set a "safe" MTU - usually to
something on the order of 1480 or smaller. But, that is incongruent with all of the
hosts that have become conditioned to expect an MTU of at least 1500. On top
of that, if the tunnel fragments at the IPv4 layer even after setting a "safe" MTU
there can be real problems. Unfortunately, after many years of study I finally
have to agree that *IPv4* fragmentation is indeed harmful.

The transitive connectivity question is an interesting one, and it may depend on
specific use cases and deployment scenarios. In enterprise networks, for example,
if A can send a tunneled packet to B, and B can send a tunneled packet to C, it does
not necessarily follow that A can send a tunneled packet to C. Chances are good
that it will work, but the best course of action is to test the path, i.e., even if you
take an initial optimistic leap of faith that the direct path works.

Thanks - Fred

> From: Owen DeLong [mailto:[email-masked]]
> Sent: Saturday, November 01, 2014 6:46 PM
> To: Iljitsch van Beijnum
> Cc: Templin, Fred L; [email-masked] WG; Keith Moore
> Why not? What's wrong with explicitly configured 6in4 tunnels? I'll grant they don't buy much vs. protocol 45 IPv6 in GRE/IPv4 tunnels,
> but there's a little less overhead and the mechanism is already defined, working, and in fairly wide implementation in various systems,
> routers, etc.
> 
> 
> >
> >> And, it need not be specific to IPv6-over-IPv4 - it would be much better to have a single mechanism that can support tunneling of
> Meh... There are tunnel mechanisms that achieve this, but they create unnecessary pain in other areas where these are not a factor.
> OTOH, I'm not sure there needs to be "one true tunneling mechanism". Requiring the additional overhead of UDP in all cases and
> STUN in general doesn't feel like a win in the cases where NAT traversal is not required.
> 
> If you think this eliminates 6in4 because some ISPs are obnoxious enough to block protocol 41, then I would suggest there is a
> reasonable likelihood that said ISPs will block whatever they have to in order to block IPv6 tunneling for the same reasons they
> blocked protocol 41.
> 
> This requires some form of service discovery mechanism that can somehow be aware of what ISP you are using or otherwise has
> some mechanism for automatically interfacing with said ISPs provisioning. In the tradition of rough consensus and running code, it
> would be great if you put forward your vision for how this would work.
> 
> > As I read this, I'm pretty sure that some of the existing tunnel broker stuff that works over UDP today already does pretty much all
> of this as far as the actual tunneling goes. What would be needed is more streamlined signup and setup processes.
> 
> I think that could be achieved as an API on top of what exists. However, see my above comments as to why I don't think that should
> be the only available solution and I definitely think that deprecation of configured 6in4 tunnels would be very premature and
> somewhat harmful in general.
> 
> > Perhaps it could look like this: as a user, you either automatically get an account using your existing email / cloud service credentials,
> or you sign up for a separate account, like [email-masked]. You enter your username and password into the home
> It's not UDP based, but FWIW there is an API to the HE tunnelbroker that was designed to allow home gateway implementers to do
> essentially just that through the web API on their gateways.
> 
> > The router or host then talks to that gateway to obtain a prefix and other configuration info and sets up the tunnel. The host or
> home gateway is then responsible for keeping NAT mappings open and reconnecting when connectivity goes away for some reason.
> >
> > I guess it would be possible for devices to log in using their MAC address by default. Not sure how much authentication we really
> need for this. An account would still be better because that way any errors could be communicated through email. And logging in
> using a known service means the user's activities aren't exposed to additional parties.
>

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Sun, 2 Nov 2014 13:31:44 +0100

If we want to come up with a tunneling mechanism that can replace 6to4 as the simple, easy choice to get tunneled IPv6 when native isn't available, perhaps it would be helpful to get together and discuss some options in Honolulu.

If anyone is interested in attending a 6to4 replacement bar BoF, please email me privately with an indication of your availability and anything else that may be pertinent.

Replying to three messages at once:

> With respect to AERO at least; the only advantage I see is that encapsulation over IPv4 instead of IPv6 saves 20 bytes per packet. I guess also that dealing with "default" is simpler than for IPv6/foo/IPv6 encapsulation.

I had a quick look at the AERO document, and it looks like AERO does much more than what's needed here. What's the intended use for AERO?

> need to learn to deal with it in new ways.

Good point. It's easy be impressed by 4% IPv6 deployment after it having been so much lower for more than a decade, but of course that still leaves 96% on IPv4.

> Or a signed certificate?

Hm, that seems pretty complex to me! As an alternative, that would be fine, but as the sole means to get a tunnel I don't think this would work well.

> give you a % that would hold true for every deployment scenario.

I didn't have transitive connectivity in mind, but rather, that if the tunnel works, it should work for all destinations, rather than the situation with Teredo (and also somewhat with 6to4) where A may be able to talk to B over the tunnel, but can't reach C because tunneled packets to C follow a different path with different NATs and firewalls.

> IPv6 ND Neighbor Unreachability Detection (NUD)

Doesn't that only kick in when there are active sessions? What I have in mind is a keepalive to determine when the tunnel has gone away (and also keeps the NAT mappings alive). Such a keepalive wouldn't have to contain an IPv6 packet but could be an empty (or nearly empty) outer header.

> You mean for mobility support? Sure.

The tunnel being torn down and a new one being created would also work.

> Gateways that terminate a tunnel, or gateways that forward tunneled packet?

Both.  :-)

> DHCPv6 PD?

Although that has the advantage of being a general purpose mechanism, I don't think it's the best solution, because now you first need the tunnel to work before you can discover the prefix that you get. If there's going to be some kind of negotiation for setting up the tunnel (which there is in what I have in mind, but not in 6to4 and many other mechanisms) then it's easy enough to incorporate prefix discovery there and reduce the number of dependencies.

> Huh? If the connection dies, the NAT mappings will be re-established when the connection is re-established.

Yes, but wouldn't it be better to avoid having the connection die?

> DHCPv6 DUID does not need to be based on MAC address.

If we require DHCPv6 then this would be an option. However, how do you know your DHCPv6 DUID? Finding a system's MAC address is easy enough.

> Why not?

Because you're limited to one proto 41 user behind an IPv4 address and thus won't work for CGN users.

> What's wrong with explicitly configured 6in4 tunnels?

I use one every day so there's a lot right with them. The trouble is that they involve too much configuration and aren't flexible enough. For instance, when I get a new IPv4 address from my ISP I need to go into the tunnelbroker.net web interface to update my tunnel endpoint.

But like I mentioned later in my message, I think much of what's required can already be found in existing tunnel broker systems, perhaps all that's needed is to bring it all together in a way that allows it to be almost as easy to use as 6to4, while continuing to benefit from the much better reliability of explicitly created tunnels.

> Meh... There are tunnel mechanisms that achieve this, but they create unnecessary pain in other areas where these are not a factor. OTOH, I'm not sure there needs to be "one true tunneling mechanis…

Even in the case where the tunnel is terminated on a home gateway and the user isn't behind a CGN, it's not uncommon for NAT to be in the middle because the tunnel endpoint is on a different box than the ISP-provided NAT box. So I'd say: eat the 8 bytes of UDP for the case of simplicity, even when it's not necessary. No need for STUN or Teredo-like complexity if the tunnel is set up from the inside to a single outside destination, you just need to generate traffic periodically to keep the NAT state in place. And you want keepalives anyway to determine when the tunnel has gone down for another reason (such as an IPv4 address change).

Another way to go would be to allow a few different types of existing tunnel mechanisms, most notably 6rd. This would make the system a little more complex on the client side, but allows for reusing existing gateway side implementations. If we go down that route, it would be easy enough to include the option for the new system to simply retrieve a fixed proto 41 configuration and be fully compatible with existing tunnel brokers.

> Harder problem to solve. I could, however, do this today with some development work on an HE tunnel.

Tunnel down -> reinitiate it from scratch. Simple enough for the client. The tunnel broker would need some way to get the new client address, though.

> Since all packet forwarding requires some level of ISP cooperation, I think you need to be a little more clear about your meaning here.

I mean in the sense that it still works if the ISP doesn't do anything to help.

It still working if the ISP goes out of its way to filter would be another step. And I don't think we should attempt that, as it would be hard to do and also because non-ISP networks such as businesses and universities have a reasonably legitimate reason to not want to have arbitrary tunnels on their networks.

Obviously we'd want those networks to offer native IPv6, but if our new tunnel mechanism can be terminated on the inside of such networks and then firewalled the same way as other traffic would also be a good solution.

> This requires some form of service discovery mechanism that can somehow be aware of what ISP you are using or otherwise has some mechanism for automatically interfacing with said ISPs provisioning.

Yes.

> In the tradition of rough consensus and running code, it would be great if you put forward your vision for how this would work.

What I have in mind is that there is a list of gateways and users are connected to the gateway that serves them best, which would typically be their ISP's gateway if they run one.

> I definitely think that deprecation of configured 6in4 tunnels would be very premature and somewhat harmful in general.

I agree; existing tunnel brokers should definitely keep doing for some time to come.

But do you agree that the current tunnel broker system is too complex to serve the role that 6to4 was intended to serve?

>> The router or host, when it notices that there is no native IPv6, connects to _tunserv._tcp.<domain> or something like that and authenticates. The setup server then does a lookup of the source IPv4 address and redirects to a gateway...

> This assumes a tremendous amount of cooperation and coordination not in evidence. Who runs this central "setup server" service? Why? How is it paid for?

What I imagine is that existing cloud services would incorporate that, most notably Apple, Google and Microsoft, who all have shown various levels of interest in getting IPv6 working well. But also any other interested party, such as the existing tunnel brokers. The technical part is basically running a web server and a user database, which doesn't cost a meaningful amount of money.

The harder part is the coordination, but the benefit is huge: you get a tunnel that's terminated as close to the user as possible without requiring any action from the user to accomplish this.

>> I guess it would be possible for devices to log in using their MAC address by default.

> There's also the issue that MAC addresses are only guaranteed to be segment unique. There are many cases where they are _NOT_ globally unique.

If the u/l bit is set to "unique" then they really are supposed to be globally unique... Obviously there's ample evidence of failures in this area, but how many affected devices are we talking about here? A fraction of a fraction of a percent?

There is no reason why one account couldn't set up multiple tunnels, so this shouldn't be an issue. The reason to have accounts at all is to be able to block them if people do bad things. Then you are blocked if you use the default account and the user of a clone of your MAC address does bad things. So set up a real account and your tunnel is back.

On 02 Nov 2014, at 6:48, Templin, Fred L <[email-masked]> wrote:

> something on the order of 1480 or smaller.

This is hard to avoid. Hopefully, with IPv6 there is enough tunneling that people will clean up their PMTUD breakage rather than let those with path MTUs below 1500 suffer, like what happens with IPv4.

A solution could be for home gateways that terminate a tunnel to advertise the tunnel MTU in their RAs so hosts use the tunnel MTU and also put that MTU in their TCP MSS.

Iljitsch

---

From: Jeroen Massar <[email-masked]>
Date: Sun, 02 Nov 2014 14:15:50 +0100

On 2014-11-02 13:31, Iljitsch van Beijnum wrote:
> If we want to come up with a tunneling mechanism that can replace
> 6to4 as the simple, easy choice to get tunneled IPv6 when native
> isn't available, perhaps it would be helpful to get together and
> discuss some options in Honolulu.

(Note that not everybody gets to take holiday trips to exotic
destinations for "work"; rather go there for vacation and enjoy the
scenery then sitting in a conference room over there...).

I don't think that "replacing" 6to4 is going to bring anything useful at
this point in time. This "transition" game has been running for close to
20 years already...

And during that time several tools have come into existence and have
gained wide deployment that solve these issues.

Teredo is the generic thing, no signup required, used heavily by Xbox
One when no native is available. But like 6to4 it has the anycast
problem: unreliable, hard to debug what goes wrong.

Tunnel Broker related (somebody needs to provide the service and keep
things working):

- heartbeat
  https://www.sixxs.net/tools/heartbeat/
  - sidechannel to proto-41 packets
  - solves the problem that people have dynamic IPv4 addresses
  - 10+ years old and heavily in use
  - as it still uses proto-41, does not work behind NAT[*]

- RFC 5572 - TSP / Tunnel Setup Protocol
  http://tools.ietf.org/html/rfc5572
  - solves setup/configuration + NAT traversal (using UDP packets)

- AYIYA
  https://www.sixxs.net/tools/ayiya/
  https://en.wikipedia.org/wiki/Anything_In_Anything
  (never went to RFC as "ipv6 WG does not do tunnels anymore" was the
   argument back then, hence also why TSP is a 'experimental' non-wg
   document)
  - solves dynamic IP + NAT traversal (signed UDP packets)

- TIC
  https://www.sixxs.net/tools/tic/
  - solves configuring a tunnel, the very simple way (linebased, no XML)

As for implementation. Both TSP and TIC/heartbeat/AYIYA can be found in
a variety of off-the-shelf hardware NAT boxes aka "consumer routers":
AVM Fritz!Box, Motorola, DrayTek etc.

Or just download the software for your favourite platform:
 - tspc (eg https://packages.debian.org/sid/tspc)
 - aiccu (https://www.sixxs.net/tools/aiccu/ or
          https://packages.debian.org/sid/aiccu)
of course available for a variety of platforms (heartbeat even for Cisco
IOS ;).

And then, I am not even going to the route of OpenVPN/tinc and thousands
of other tunneling techniques that can carry IPv6 packets inside IPv4
and can do VPNs.

What part of '6to4 exactly needs a replacement while all these things
have been available for years already?

Greets,
 Jeroen

[*] setting your NAT box to DMZ mode so that a specific host gets all
packets and that that host terminates the tunnel is an option. Not all
users get that though and not all boxes handle it properly...

---

From: "Templin, Fred L" <[email-masked]>
Date: Sun, 2 Nov 2014 13:44:52 +0000

Hi Iljitsch,

Don't be too quick to write off AERO. AERO is NBMA - you should like that, because
it has a lot in common with Ethernet except for lack of link-scoped multicast. Think of
your customer tunnel endpoints as AERO Clients and your provider tunnel endpoints
as AERO Servers. Route optimization gives you Client-to-Client. DHCPv6 PD is the control
plane signaling between Clients and Servers, and IPv6 ND is the control plane signaling
between pairs of Clients.

About MTU:

> This is hard to avoid. Hopefully, with IPv6 there is enough tunneling that people will clean up their PMTUD breakage rather than let
> those with path MTUs below 1500 suffer, like what happens with IPv4.

Not if you use tunnel fragmentation and probing. This is specifically NOT IPv4-layer
fragmentation, NOT IPv6-layer fragmentation and NOT RFC4821 probing - it is a private
arrangement between the tunnel ingress and egress that is not visible at the inner or
outer IP layer. An assured minimum of 1500 bytes is therefore provided, and if larger
packets can fit they will be accommodated as well.

> and also put that MTU in their TCP MSS.

This can have a cascading degenerate MTU problems if there are tunnels within tunnels
inside the site. Use a real tunnel MTU mitigation, and the site will never see an MTU
smaller than 1500 (i.e., a degenerate MTU).

Thanks - Fred

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Sun, 2 Nov 2014 16:17:21 +0100

> this point in time. This "transition" game has been running for close to

I agree somewhat. But the real question is how much longer it's going to take rather than how long it's been under way already... Native IPv6 deployment is picking up speed, but on the other hand even under the most optimistic assumptions it's going to take at least another half a decade to reach ubiquity.

> gained wide deployment that solve these issues.

Compared to 6to4 and Teredo? I don't think so. And those two have significant issues that make them as good as unworkable.

> - RFC 5572 - TSP / Tunnel Setup Protocol

You missed a few.  :-)  See RFC 7059.  (-:

You and Owen make a good point that existing tunnel brokers already do good work, and of course adding one new protocol to rule them all hasn't historically been very successful.

But... wouldn't it be great if we could have a tunnel mechanism that couples the reliability of a tunnel broker tunnel with the ease of use (i.e., it's enabled by default without the user having to do anything) of 6to4/Teredo?

I think it could be done with a simple front end that allows a home gateway (or a host, of course) to obtain configuration info so it can connect to an appropriate tunnel endpoint.

The actual encapsulation is easy enough to negotiate: let the client indicate whether it can handle plain proto 41, 6rd, IPv6 in UDP, AYIYA, AERO, whatever. (Of course one would have to be the "lingua franca".) The server can then see if the user is signed up to any services that provide compatible tunnels, if the client's IP address map to a service provider tunnel gateway or if there is a public gateway that serves those addresses and then provides the provisioning details.

This would probably require some back end changes in places like tunnelbroker.net and sixxs.net, but it would otherwise let users continue to use those services but with the benefit that if home gateway makers implement the new setup protocol, the tunnels can be terminated using those home gateways, which makes for a much better user experience than doing it on a computer. And because everything is 

Iljitsch

---

From: "Templin, Fred L" <[email-masked]>
Date: Sun, 2 Nov 2014 15:28:15 +0000

> This would probably require some back end changes in places like tunnelbroker.net and sixxs.net, but it would otherwise let users
> continue to use those services but with the benefit that if home gateway makers implement the new setup protocol, the tunnels can
> be terminated using those home gateways, which makes for a much better user experience than doing it on a computer. And because
> everything is

I say turn the tunnel brokers into DHCPv6 servers and the home gateways into DHCPv6 clients.
Let them use DHCPv6 PD and DHCPv6 security to make sure that only authorized clients get
IPv6 prefix delegations. Then, layer an NBMA interface model on top of everything and use
IPv6 ND to coordinate route optimization and mobility management. It just might get us to
a fully IPv6 and fully mobile Internet...

Thanks - Fred

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Sun, 2 Nov 2014 16:37:41 +0100

> I say turn the tunnel brokers into DHCPv6 servers and the home gateways into DHCPv6 clients.

That doesn't work, because you need the remote tunnel endpoint configuration info before you can talk to the tunnel broker over IPv6 so you can send your DHCPv6 request in order to get your configuration info...

Apart from that, I'm not a huge fan of putting everything in DHCP(v6) because DHCP is hard to get at from the outside. Life is so much easier if you can simply use HTTP.

---

From: "Templin, Fred L" <[email-masked]>
Date: Sun, 2 Nov 2014 15:50:14 +0000

> Sent: Sunday, November 02, 2014 7:38 AM
> To: Templin, Fred L
> Cc: Jeroen Massar; Owen DeLong; [email-masked] WG
> Subject: Re: [v6ops] Bar BoF on a 6to4 replacement? Was: my recommendation re: 6to4
> 
> That doesn't work, because you need the remote tunnel endpoint configuration info before you can talk to the tunnel broker over
> IPv6 so you can send your DHCPv6 request in order to get your configuration info...

The tunnel broker address is automatically derived from the DNS. However, the client does
need to be enrolled in the service offered by the tunnel broker in order to get an IPv6
prefix delegation. Is that any different than what current tunnel broker services require?

> Apart from that, I'm not a huge fan of putting everything in DHCP(v6) because DHCP is hard to get at from the outside.

I'm struggling to understand what that means: "hard to get at from the outside"?

Thanks - Fred

> Life is so much easier if you can simply use HTTP.

---

From: Jeroen Massar <[email-masked]>
Date: Sun, 02 Nov 2014 17:58:53 +0100

On 2014-11-02 16:17, Iljitsch van Beijnum wrote:
>> I don't think that "replacing" 6to4 is going to bring anything
>> useful at this point in time. This "transition" game has been
>> running for close to 20 years already...
> 
> I agree somewhat. But the real question is how much longer it's going
> to take rather than how long it's been under way already... Native
> IPv6 deployment is picking up speed, but on the other hand even under
> the most optimistic assumptions it's going to take at least another
> half a decade to reach ubiquity.

ISPs who are not moving will not move, the will wait till their
customers move to another.

Users who want IPv6 have already for a decade been choosing to use
Tunnel Brokers.

Users who do not care, well, they do not care.

>> And during that time several tools have come into existence and
>> have gained wide deployment that solve these issues.
> 
> Compared to 6to4 and Teredo? I don't think so.

You don't think what part?

> And those two have
> significant issues that make them as good as unworkable.

You mean that 6to4 + Teredo have issues I guess?

>> - heartbeat - RFC 5572 - TSP / Tunnel Setup Protocol - AYIYA - TIC
> 

I don't miss anything significant as most of those are not used anywhere.

6in4 = See [1], 10k at SixXS, probably another 10k at HE (they do not
report active tunnels, just 'count', thus unknown what that means active
or 'configured and not used'?), what every TB does, hence definitely
used a lot.

"Automatic Tunneling" nothing uses that.

6over = nothing does that

GRE = happens sometimes (and actually, because some platforms do GRE but
not 6in4 there is this sixxsd beta with GRE tunnels, not much different
protocol from 6in4 anyway :)

6to4 = the one that has to go away

AYIYA = 25k tunnels, see [1], used a *lot*

ISATAP = maybe some "enterprise", but outside of that very little

Teredo = the other anycast thing (Xbox One primarily + BitTorrent)

6rd = lots of ISP-pushed deployment; 6in4 based

6a44 = never even seen code or deployment

LISP = just some people playing test setup, no real deployment

SEAL = never seen code or deployment

6bed4 = never seen code or deployment

TSP = About 10k tunnels:

http://go6.net:3653/graph_image.php?action=view&local_graph_id=547&rra_id=1

[1] = https://www.sixxs.net/misc/usage/ as a general sample of how these
are used.

> You and Owen make a good point that existing tunnel brokers already
> do good work, and of course adding one new protocol to rule them all
> hasn't historically been very successful.

As per [1] above, 6in4 is only 25% of the SixXS user base, >50% is AYIYA
simply because of the NAT piercing.

If you can do 6in4 directly you are lucky to have a public IP or are
able to force that IP to become exposed with "DMZ mode" or similar.

> But... wouldn't it be great if we could have a tunnel mechanism that
> couples the reliability of a tunnel broker tunnel with the ease of
> use (i.e., it's enabled by default without the user having to do
> anything) of 6to4/Teredo?

You can't.

One of the very much overseen problems with running a Tunnel Broker is
abuse handling. SixXS 'solved' this by asking for personal details and
verifying these and pouncing hard on abuse, thus leaving the others
Tunnel Brokers to handle the people who like mischief.

> I think it could be done with a simple front end that allows a home
> gateway (or a host, of course) to obtain configuration info so it can
> connect to an appropriate tunnel endpoint.

As noted quite a few products out there have TIC or TSP implemented
already, eg:

https://www.sixxs.net/wiki/Fritz!Box_general or directly the important
picture:

https://www.sixxs.net/wiki/File:FritzboxHowto.jpg

Oh, and yes, if Hurricane Electric wanted to use that protocol they
could have taken that feature a long time ago too and have it. Nothing
tricky about it, solves exactly what it needs to solve.

> The actual encapsulation is easy enough to negotiate: let the client
> indicate whether it can handle plain proto 41, 6rd, IPv6 in UDP,
> AYIYA, AERO, whatever.

The client does not know that. Also note that more and more people
become mobile and switch connection / SSID mid-session.

Just using AYIYA solves all of that.

The sixxsd mentioned above that does GRE, and thus beta, does allow
automatic switching between GRE/proto-41/AYIYA though using the signed
heartbeat to verify that the source is really that source.

> (Of course one would have to be the "lingua
> franca".) The server can then see if the user is signed up to any
> services that provide compatible tunnels, if the client's IP address
> map to a service provider tunnel gateway or if there is a public
> gateway that serves those addresses and then provides the
> provisioning details.

TIC handles all of that already.

> This would probably require some back end changes in places like
> tunnelbroker.net and sixxs.net, but it would otherwise let users
> continue to use those services but with the benefit that if home
> gateway makers implement the new setup protocol, the tunnels can be
> terminated using those home gateways, which makes for a much better
> user experience than doing it on a computer. And because everything
> is

TIC is already implemented in these machines. Go shop around for AVM
Fritz!Box, Draytek, ZyXEL, Motorola and various more, lots of them have
added 'aiccu' to their setup as they are just Linux boxes anyway and as
AICCU is BSD-licensed, they can do so without any problems.

Some support just proto-41/heartbeat, others also have AYIYA support.

And there are also boxes that support TSP, next to Hexago/Go6/Gogo6
having their GoGoCPE: http://www.gogo6.com/gogoware/gogocpe that is a
mini-CPE that does the work for you for a small amount of $$$.

Greets,
 Jeroen

---

From: Jeroen Massar <[email-masked]>
Date: Sun, 02 Nov 2014 18:01:47 +0100

On 2014-11-02 16:50, Templin, Fred L wrote:
> The tunnel broker address is automatically derived from the DNS.

Or you just get those details from a protocol like TIC or TSP and let
the actual Tunnel Broker decide on these details.

> However, the client does need to be enrolled in the service offered by the tunnel broker in order to get an IPv6

No difference at all.

But there is absolutely no reason to do DHCPv6.
Both SixXS and HE do static prefix delegation.

Per default there is a _routed_ /64, and otherwise you request a large one.

As the user gets a _static_ prefix, if somebody wants to bother with
DHCPv6, let them do that on their end.

Greets,
 Jeroen

---

From: "Templin, Fred L" <[email-masked]>
Date: Sun, 2 Nov 2014 17:26:11 +0000

Hi Jeroen,

> From: Jeroen Massar [mailto:[email-masked]]
> Sent: Sunday, November 02, 2014 9:02 AM
> To: Templin, Fred L; Iljitsch van Beijnum
> Cc: Owen DeLong; [email-masked] WG

Well, looking up a FQDN to get A or AAAA records gives you some assurance that
the tunnel broker you are talking to has properly registered itself in the DNS.

OK.

> Both SixXS and HE do static prefix delegation.

In a sense, it is essentially static prefix delegation with the AERO DHCPv6 service
model as well since the service needs to maintain a table of Client ID-to-prefix
mappings.

> But there is absolutely no reason to do DHCPv6.

Mobility and multihoming. The Client can register itself with the Server using whatever
access network IP address(e) it happens to have at the moment. So, for example, if the
Client gets a different access network IP address every time it boots up the Server learns
the new address during the DHCPv6 Request/Reply exchange. Afterwards, if the Client
gets a new access network IP address (e.g., when moving from 4G to WiFi) it issues a
DHCPv6 Rebind and the Server updates its address. Or, if the Client has multiple active
access network IP addresses, it can register them all with the Server using DHCPv6
Rebind and again use Rebind if the addresses ever change. 

> Per default there is a _routed_ /64, and otherwise you request a large one.

Prefix lengths other than /64 should be supported, yes. And, they must be
routable as well - no difference there.

> DHCPv6, let them do that on their end.

A Client that receives a prefix via DHCPv6 PD from a Server can certainly turn around
and act as a DHCPv6 server to sub-delegate portions of that prefix to its downstream
attached end user networks, yes. That is the same as for any DHCPv6 PD scenario.

Thanks - Fred

---

From: Jeroen Massar <[email-masked]>
Date: Sun, 02 Nov 2014 19:21:40 +0100

On 2014-11-02 18:26, Templin, Fred L wrote:
[..]
> the tunnel broker you are talking to has properly registered itself in the DNS.

As the TIC server is identified with a DNS label, you do that.

(And you do not want a AAAA record when you do not have IPv6
connectivity yet)

> model as well since the service needs to maintain a table of Client ID-to-prefix

A tunnel broker really can't care less about DHCP, it is all static.

Actually, except for the current IPv4 endpoint, an identity and a
passphrase, there is very little state in a proper tunnel broker.

> Mobility and multihoming.

You do not need DHCPv6 for either of that.

Note that heartbeat and AYIYA "solve" (for varying degrees of "solve")
Mobility aka "my IPv4 endpoint changes" or better "my current IPv4
endpoint and optionally UDP port are X and Y).

> The Client can register itself with the Server using whatever
> the new address during the DHCPv6 Request/Reply exchange.

Why bother with DHCP? You only need to signal the current IPv4 endpoint:
heartbeat or AYIYA (which does a per-packet heartbeat).

> Afterwards, if the Client

No need to do "Rebinds" or other stuff like that. Every heartbeat causes
the endpoint to be updated.

You are trying to make it all to Enterprisey and solving problems that
are not there.

People who have multiple IPv4 endpoints will use the protocols that are
suited for that.

6in4/heartbeat/AYIYA/TSP are all *transition technologies* they do not
solve multihoming and other such things.

AYIYA could solve those, but I chose not to bother with it. Very few
people have multiple upstreams, and if you do, you can chose your
'current' one by sourcing the packets from there.

> routable as well - no difference there.

Per default it is a /64 so that the enduser can announce that per radvd
on their local LAN.

If they want to request "a large one" which typically is a /48.

As it is just a line in TIC, it does not matter what happens to it or
how many bits it has. As it is static, they do not even have to look it
up in TIC actually. Though Fritz!Box and Motorola do that as then the
user does not have to type anything except user/pass/tunnel_id.

> attached end user networks, yes. That is the same as for any DHCPv6 PD scenario.

The user can chose themselves to bother with DHCP if they want that
state stuff. For the tunnel and prefix delegation it is not needed.

I actually have myself never bother with DHCPv6, I just do static radvd,
simple and works. I do hear a lot of people have problems with it. If
people want to play with it, they can, their network on that side of the
tunnel. No need to get Tunnel Brokers to do so.

For SixXS we did think of doing DHCPv6-PD but decided that it is too
much overhead and state for something that is static anyway. Why have
state when one does not need it?

And I actually don't recall anybody asking about such a 'feature';
likely as there is no reason for Tunnel Brokers to do so anyway.

Greets,
 Jeroen

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 03 Nov 2014 08:34:46 +1300

> gained wide deployment that solve these issues.

Yes. When I don't find native v6 I either lie back and enjoy NAT44
or I hook up to SixXs. It's actually hard to beat AYIYA except that
the setup is for geeks rather than normal humans. (The lack of an
RFC doesn't seem to matter, which rather puts the discussion of
deprecation and Historic status for 6to4 in its proper perspective.)

I think this topic is OBE. We should focus on the ISPs and content
providers who didn't get the message.

   Brian

> _______________________________________________
> v6ops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/v6ops
> .
>

---

From: "Templin, Fred L" <[email-masked]>
Date: Sun, 2 Nov 2014 21:21:59 +0000

Hi Jeroen,

You seem to be getting wrapped up with the notion that I am wanting DHCPv6 to maintain
lots of unnecessary state. I don't - I am primarily interested in the DHCPv6 signaling messages
themselves and their implication for the NBMA interface neighbor cache:

- DHCPv6 Request/Reply creates a neighbor cache entry with the Client's AERO address
  as the network-layer address and the Client's access link address as the link-layer address.

- DHCPv6 Renew/Reply refreshes the neighbor cache entry lifetime

- DHCPv6 Rebind/Reply either adds/deletes a new access link (there may
  be multiple of these) or changes the link-layer address of an existing
  access link

- DHCPv6 Release/Reply is for shutting down the service or moving to a new Server

Try not to think of this as me getting "enterprisey", except that in this case the
"enterprise" is the entire public Internet. I know the document currently says
"enterprise" in a lot of places, but let's try to keep an open mind about other
possibilities.

Thanks - Fred

> Sent: Sunday, November 02, 2014 10:22 AM

---

From: Jeroen Massar <[email-masked]>
Date: Mon, 03 Nov 2014 07:42:57 +0100

> themselves and their implication for the NBMA interface neighbor cache:

For the subject at hand:
 "getting IPv6 to endsites that do not have it yet"

You do not need anything DHCP-ish. DHCP is for directly-on-the-same-link
endpoints. Tunnels are not on-link, they are
'somewhere-on-the-internet'. And you need a tunnel before being able to
do anything DHCP-ish. Unless you propose people start dropping those
DHCP firewall rules around the Internet (yes, people block DHCP there
for obvious reasons).

TSP or TIC (pick your poison) already solve the "signaling" for figuring
out the tunnel parameters. Then use 6in4(-heartbeat)/TSP/AYIYA to
actually get the tunnel going.

No need to come up with new protocols that make things more complex than
that.

TSP (the signaling/configuration part) is already XML based for
'extensibility'.

Note that TIC was developed in parallel and chose a simple SMTP-alike
protocol; though if I would have to do it now I would simply use HTTP as
that has a lot less issues with firewalls.

Greets,
 Jeroen

---

From: Jeroen Massar <[email-masked]>
Date: Mon, 03 Nov 2014 08:27:50 +0100

On 2014-11-02 20:34, Brian E Carpenter wrote:
> the setup is for geeks rather than normal humans.

Yep, an AICCU installer would be something quite useful. Something with
time and other things to do unfortunately.

On Debian it is just 'apt-get install aiccu', on OSX it is a 'brew
install aiccu' and one is done though. Windows changes also a lot
needing new tun/tap drivers quite often which makes it a bit more
time-required to maintain.

> (The lack of an

They typically have business cases clashing with IPv6 (not wanting to
give out IP addresses due to 'servers' that clashes with their hosting
business), or lazy folks who didn't pay attention and thus did not
calculate this whole 'upgrade hardware to a device that supports IPv6'
in the last 15 years... and thus are waiting for the next replacement
round and just sitting it out.

There is:
 - native, for when the hardware supports it
 - 6rd, for when the last-leg hardware does not

Anything else is for the end-user to do, eg tunnel broker.

I really cannot see how the IETF can do more (in the way of protocols)
that could further help this situation.

Note that I have heard ISPs who are deploying DSLite (thus providing
native IPv6 and CGN IPv4) pointing their customers at SixXS so that the
end-user can regain access to their home network from their
$otherlocation. That is one of the fun counters of DSLite.
Then again, users from that same ISP have been doing AYIYA over the CGN
so that they get a static IPv6 prefix and apparently AYIYA is faster
over the CGN than the native IPv6 that ISP provides, which is a ehmm
funny...

Greets,
 Jeroen

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 3 Nov 2014 16:21:01 +0000

Hi Jeroen,

> Sent: Sunday, November 02, 2014 10:43 PM
> To: Templin, Fred L
> Cc: [email-masked] WG
> endpoints.

For the NBMA tunnel virtual interface I am describing, all endpoints are
directly-on-the-same-link, and DHCPv6 clients and servers are always
single-hop neighbors.

> Tunnels are not on-link, they are 'somewhere-on-the-internet'.

Again, all endpoints on the NBMA tunnel virtual link are on-link. DHCPv6 and
IPv6 ND works over the link the same as for any IPv6 link.

> And you need a tunnel before being able to do anything DHCP-ish.

The NBMA tunnel interface is always available, and can be used to send an
encapsulated packet to any prospective neighbor at any time and with no
explicit configuration. Brian Carpenter knows this. He invented it.

> Unless you propose people start dropping those

It would not look like DHCP on the physical links that carry the tunneled packets;
it would all just look like UDP port 8060.

> actually get the tunnel going.

There is no need to get an NBMA tunnel going; it is always going, and always
available to send tunneled packets to any prospective neighbor.

> No need to come up with new protocols that make things more complex than

You are inferring complexity where there is none. Any Client can send a
Server an encapsulated DHCPv6 or IPv6 ND packet at any time and with
no explicit tunnel configuration. And, it is really not a new protocol; it
is based on long-standing Internet standards.

> that has a lot less issues with firewalls.

AERO uses IPv6 ND and DHCPv6 the same as for any link. This takes care of
neighbor discovery, mobility management, multihoming, link-local address
autoconfiguration, prefix delegation and anything you might want to do on
an ordinary IPv6 link with one exception - the AERO link is link-local-only.

Thanks - Fred

---

From: Jeroen Massar <[email-masked]>
Date: Mon, 03 Nov 2014 18:22:00 +0100

> Hi Jeroen,

So it seems we strayed somewhere off the '6to4 replacement' subject and
got into a discussion about AERO / RFC6706. Apologies that I did not
notice that context switch. Updated the Subject: to match this.

> directly-on-the-same-link, and DHCPv6 clients and servers are always

If you indeed have a such a virtual network already setup, then as it is
multi-access doing DHCP makes sense.

The problem is that you first need to set up such an (overlay) network
and inform all involved who are involved.

[..]
> explicit configuration. Brian Carpenter knows this. He invented it.

As Brian has a very large curriculum, which one do you mean?

Encapsulating things does not mean that those packets make it to the
other side. proto-41 for instance does not go through NATs.

Something has to set it up, something has to configure it.

> it would all just look like UDP port 8060.

Aha, it seems you are talking about experimental RFC6706.

The only reference in that document to using the port is the "AERO
Redirection Message Format", thus I can only assume that something else
has to carry the actual packets and those packets are not likely going
to go over the 8060/udp port you mention.

Something has to tell the involved hosts where and how to send those
packets though, there is no magic specified in that RFC that will tell
otherwise. Those parameters are part of your tunnel setup.

And even if those special "AERO" packets are inside UDP, where are the
real packets going to be encapsulated in?

> available to send tunneled packets to any prospective neighbor.

Something has to configure it; after that, yes, likely that it will keep
on working. Though NATs have these nasty timers that this magic NBMA
tunnel has to account for and keep alive (simply keeping on sending
packets typically does the trick though, but one has to account for the
source port on the NAT side changing over time and that typically this
only works for UDP).

> is based on long-standing Internet standards.

That can only work when you have that virtual overlay that can do
multicast style setups.

It will be grossly inefficient to that on a scale of 10.000 clients
where only 2 (the server and that specific client) need to see that packet.

As such not appropriate for a system the scale that should 'replace' 6to4.

> AERO uses IPv6 ND and DHCPv6 the same as for any link.
> This takes care of neighbor discovery

Why does a tunnel need Neighbor Discovery? Unless you are stuffing every
client in the world on the same /64. The endpoints likely already know
what the other end is anyway.

> mobility management

Mobility of what kind? There is all kinds of mobility.

Some people still think that IPv6 Mobility is an actual thing that
exists and gets used. I have actually never seen anybody use it... KAME
MIP6 etc are there but rarely actually get used except for
playing/testing. http://www.mobileip.jp/ is even missing in action...

> multihoming,

Single homing right? There is no "Send packets here first" or "send 50%
of the packets here" or "I am also at" messages in AERO from what I can see.

Hence, even if we just look at the redirect, the packet will go to one
place. Maybe you mean that the underlying carrier handles this?

Proper multihoming has a lot of factors, amongst which available
bandwidth per endpoint, latency properties etc. None of which are easily
figured out and most of which are easily misguessed/configured.

Hence why most multihoming setups typically monitor the availability of
a link and then just hot-potato packets over each link, stopping to send
when that link is 'full' or does not respond in an expected way.
Or by selecting based on QOS or specific destinations. BGP/OSPF etc come
into play a lot there.

> link-local address autoconfiguration,

The link-protocol (Ethernet or other tunnel mechanism) should take care
of that. Thus all link types that require it will do that. How is this a
special feature?

> prefix delegation and anything you might want to do on

Why would it only be link local?

Anyway, my apologies for not reading into your messages that you where
talking about RFC6706. More apologies for not knowing that it was a
draft and was going for RFC otherwise I could have read and reviewed it
and asked various questions about it.

For me RFC6706 is a long piece of text with a lot of claims and
complexity, but it does not actually state what is actually happening.

>From reading it my summary is:
 - AERO defines a 'redirect' message to 'repoint a tunnel endpoint'
   (section 6.4.5)
 - some tunneling protocol is used to carry actual packets.
 - not defined how these tunnels are configured

As such, for me that whole RFC is unclear what the target is, or more
importantly what problem it is supposed to solve in the first place.
(yes, there are requirements, but these seem to be written around what
the protocol does, not what actual problems there are in the world)

And thus I also wonder how that got into this thread about a 6to4
replacement: problem = get IPv6 to endhosts that do not have it.

But then again, I might just be missing something completely obvious...

Greets,
 Jeroen

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 3 Nov 2014 17:40:30 +0000

Hi Jeroen,

> Sent: Monday, November 03, 2014 9:22 AM
> To: Templin, Fred L
> Cc: [email-masked] WG
> Subject: Seems AERO / RFC6706 is the subject (Was: [v6ops] Bar BoF on a 6to4 replacement? Was: my recommendation re: 6to4)
> 

The correct reference for this discussion is 'draft-templin-aerolink', which will obsolete
RFC6706 when it is published. RFC6706 can even now be considered as historic even
though it has not been officially designated as such.

However, 'draft-templin-aerolink' is also a candidate replacement for 6to4. AERO
works everywhere AFAICT, and does more than just a transition mechanism would.
It is in fact intended as a long-term solution which has IPv6/IPv4 coexistence as just
one aspect. Mobility, multi-homing, route optimization, security, etc. are just a few
of the other aspects.
 
> and inform all involved who are involved.

There is no setup needed. It is tunneling without explicit tunnels.

> As Brian has a very large curriculum, which one do you mean?

RFC2529 (circa 1999).

> other side. proto-41 for instance does not go through NATs.

AERO can work over proto-41, but the normal case is to use UDP port 8060.

> Something has to set it up, something has to configure it.

It is a tunnel without explicit tunnel configuration.

> Aha, it seems you are talking about experimental RFC6706.

No; I am talking about 'draft-templin-aerolink'. UDP port 8060 is shared between
RFC6706 and 'draft-templin-aerolink'.

> to go over the 8060/udp port you mention.

We are out of context; the correct reference for this discussion is
'draft-templin-aerolink'. Although they are much appreciated, I don't think
it would be helpful to try to respond to your other points for now until we
get into the correct context.

Thanks - Fred

---

From: Jeroen Massar <[email-masked]>
Date: Mon, 03 Nov 2014 19:37:17 +0100

On 2014-11-03 18:40, Templin, Fred L wrote:
[..]
> one aspect. Mobility, multi-homing, route optimization, security, etc. are just a few

That is one very very big document with a lot of funny technical constructs.

But it misses a very important section "Problem Statement":
  What is the problem it is trying to solve that cannot be solved
  already with other tools/protocols?

[..]
> There is no setup needed. It is tunneling without explicit tunnels.

What tool tells the AERO components what "AERO Service Prefix (ASP)" to
use? What "AERO Server" to talk too etc? Where does that come from?

What are the authentication parameters? This Internet thing is quite
evil. Accepting packets from random sources will not fly for long.

[..]
>> As Brian has a very large curriculum, which one do you mean?

aka "6over" which is afaik not used anywhere?

Something with "there is no multicast" especially not on the Internet.
Nope, university networks do not count and typically that is only the
core which gets it, if they are 'lucky'.

> AERO can work over proto-41, but the normal case is to use UDP port 8060.

Just putting packets in UDP is not enough. You need to make sure that it
can handle a switching source UDP port and that there are enough packets
being sent to keep the binding open.

Also, such a tunnel is per definition a point-to-point link. That the
routing protocols above might flood packets to all those links is a
separate thing.

> It is a tunnel without explicit tunnel configuration.

See above, you have parameters, these need to be configured.

> RFC6706 and 'draft-templin-aerolink'.

I was not aware of the latter; will read more if you can specify the
problem statement and how this is supposed to solve those problems.

> get into the correct context.

Subject is fixed to solve that problem

Greets,
 Jeroen

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 3 Nov 2014 19:19:42 +0000

Hi Jeroen,

> Sent: Monday, November 03, 2014 10:37 AM
> To: Templin, Fred L
> Cc: [email-masked] WG
> Subject: Re: AERO / draft-templin-aerolink / RFC6706
> 

Full routing and addressing support for fixed, mobile and/or multi-homed devices in an
Internetwork using IPvANY-in-IPvANY encapsulation. Example Internetworks include
enterprise networks, cellular operator networks, ISP networks, and the public Internet
itself. As the SixXS POC, you might also consider using AERO to provide IPv6 services to
Client customers to give them more than just IPv6/IPv4 transition. The way it would work
is that the SixXS tunnel broker(s) would be replaced by AERO Servers - of which there may
be many (e.g., 10's, 100's, etc.) topologically distributed throughout the Internet.  Clients
get services from a nearby Server, and Clients get the same services no matter which
Server they associate with. So, the Client load is spread. If the Client gets a new
link-layer address (e.g., if it needs to blast a new hole through an existing NAT, if
changes from 4G wireless to WiFi, etc.) it sends DHCPv6 Rebind to its Server which
updates its neighbor cache link-layer address.

> use?

DHCPv6.

> What "AERO Server" to talk too etc? Where does that come from?

DNS lookup for the domain name "linkupnetworks.example.com" returns a
list of link-layer addresses of candidate nearby AERO Servers. "example.com"
is a domain name managed by the AERO Service Provider.

> evil. Accepting packets from random sources will not fly for long.

Secure DHCPv6 (Clients sign their messages based on certificates).

> aka "6over" which is afaik not used anywhere?

I should have been more specific. The aspect of Brian's invention that carries
forward into AERO is the point-to-multipoint tunnel virtual interface model,
where there is no explicit configuration needed for a node to send a tunneled
*unicast* packet to an on-link neighbor.

> core which gets it, if they are 'lucky'.

Again, I should have been more specific. Unlike RFC2529, the AERO tunnel
virtual interface model is NBMA in its basic manifestation. However, there
is support for at least "All_DHCP_Relay_Agents_And_Servers" multicast and
there is a section on how to support other multicast, based on whether the
underlying network is multicast-capable or non-multicast-capable. So, consider
NBMA as the basic service model and partial/full multicast-capable as the
"augmented" service model.

> being sent to keep the binding open.

AERO sends periodic NS/NA (or RS/RA) to keep middlebox state alive.
If the UDP source port changes, the path to the neighbor will appear
to be failing and the Client can either try to do DHCPv6 Rebind with the
existing Server or ditch the existing Server and associate with a new
Server. (The spec doesn't currently talk about Rebind, though; it would
probably be worth fixing this because frequent changes between Servers
is not a very good idea.)

(BTW, the UDP port is considered as part of the Client's link-layer address.
If the UDP port changes, then the Client's link-layer address changes and
needs to be updated with any existing neighbors. This is indistinguishable
from  a mobility event.)

> Also, such a tunnel is per definition a point-to-point link. That the

Not a point-to-point link, rather there is an NBMA interface neighbor
relationship between the two neighbors that may require periodic "bubbles"
(teredo terminology) or "heartbeats" (your terminology, I think?) to keep
neighbor and middlebox state alive. For AERO, that is typically NS/NA.
 
> routing protocols above might flood packets to all those links is a

I don't get the part about routing protocol flooding; that doesn't resonate
with any aspects of AERO that I am aware of. Can you be more specific?

> See above, you have parameters, these need to be configured.

The parameters are just neighbor cache entry network- and link-layer addresses.
These are autoconfigured during DHCPv6 exchanges.

> problem statement and how this is supposed to solve those problems.

Let me now if you need more information than I have already given.

OK.

Thanks - Fred

---

From: Owen DeLong <[email-masked]>
Date: Mon, 3 Nov 2014 13:11:47 -0800

I won’t be in HNL, so I can’t attend the bar BoF.

> Because you're limited to one proto 41 user behind an IPv4 address and thus won't work for CGN users.

So what you’re really saying here isn’t “MUST NOT be protocol 41” so much as “MUST BE use a known NAT-Friendly Transport Layer Protocol.”

> I use one every day so there's a lot right with them. The trouble is that they involve too much configuration and aren't flexible enough. For instance, when I get a new IPv4 address from my ISP I n…

Sure, but a little bit of software added to your router using the API that HE publishes for that _COULD_ actually automate that.

> But like I mentioned later in my message, I think much of what's required can already be found in existing tunnel broker systems, perhaps all that's needed is to bring it all together in a way that…

Yep.

My point is that HE has actually already built all of the hooks necessary on their side. All that is needed is for router and/or host vendors to write the client-side glue.

There is a published API for the tunnel broker.

Admittedly, that doesn’t work for NAT or CGN users because the HE tunnelbroker is protocol 41 based, but it does cover the vast majority of cases.

Do you really expect vast numbers of CGN implementations that don’t provide native IPv6 at this point? Most of the implementations of CGN that I’m aware of are targeted at supporting IPv6 users that still need IPv4 access.

> Even in the case where the tunnel is terminated on a home gateway and the user isn't behind a CGN, it's not uncommon for NAT to be in the middle because the tunnel endpoint is on a different box th…

That requires some form of reliable way to make sure that all return traffic is routed back through the same gateway at the other side. This is obviously achievable, but it reduces reliability or increases complexity.

> Another way to go would be to allow a few different types of existing tunnel mechanisms, most notably 6rd. This would make the system a little more complex on the client side, but allows for reusin…

I think this is probably the best way to go.

> Tunnel down -> reinitiate it from scratch. Simple enough for the client. The tunnel broker would need some way to get the new client address, though.

Actually, I think the client should expect to inform the tunnel broker of its address change…

1.	These are intended to be stateless tunnels, so authentication isn’t feasible because authenticated vs. unauthenticated _IS_ state.
2.	The client can easily know when its address has changed. OTOH, other than by address, it’s hard for the tunnel broker
	to know, and if the address no arriving is unknown, virtually impossible to know what tunnel to associate it with and update.

Currently, using existing APIs, the client, upon detecting a new address could access and reconfigure the Tunnel Broker and the tunnel would come back up. I had, at one point, scripted most of this on my site, but it was fragile because the script depended on polling my router from a third host and then connecting to the tunnel broker OOB.

> I mean in the sense that it still works if the ISP doesn't do anything to help.

Sure, on this I agree (presuming that simply forwarding packets is within the bounds of not doing anything to help), but yes, I agree in the context of “ISP shouldn’t have to host or participate in configuring or provisioning tunnels or providing tunnel termination services in order for it to work”.

> It still working if the ISP goes out of its way to filter would be another step. And I don't think we should attempt that, as it would be hard to do and also because non-ISP networks such as busine…

Agreed.

Instead, we should simply attempt to destroy such ISPs through peer pressure. ;-)

> Obviously we'd want those networks to offer native IPv6, but if our new tunnel mechanism can be terminated on the inside of such networks and then firewalled the same way as other traffic would als…

Ideally, yes, but doing this (firewalling tunneled traffic) in an IPv6-ignorant network is difficult at best.

> What I have in mind is that there is a list of gateways and users are connected to the gateway that serves them best, which would typically be their ISP's gateway if they run one.

How would this list be maintained? What would be the bar for getting on the list? What would be the mechanism for maintaining the list? How would a device locate the list at bootstrap?

This is a nice hand wave and I understand the general desired outcome, but there are many devils in the details.

That’s what I was trying to convey with the “running code” part of my comment.

> But do you agree that the current tunnel broker system is too complex to serve the role that 6to4 was intended to serve?

I’ve never really been completely clear on the intended role for 6to4 since it so clearly missed whatever mark that was, so it’s hard to comment authoritatively.

I will say that in my experience, no, most of the tunnel broker systems are not that hard to use. What they lack is sufficient automation to achieve the self-configuring aspects of 6to4. IMHO, said automation could be built onto the existing services without too much effort. What I don’t get is why none of the router vendors that set out to do this have followed through.

> What I imagine is that existing cloud services would incorporate that, most notably Apple, Google and Microsoft, who all have shown various levels of interest in getting IPv6 working well. But also…

That depends on the level of support you want to provide to the users when something goes horribly wrong.

That can cost anywhere from virtually nothing to very much.

> The harder part is the coordination, but the benefit is huge: you get a tunnel that's terminated as close to the user as possible without requiring any action from the user to accomplish this.

I don’t think that tunnels that don’t require ANY action from the user are a good thing, actually.

The user _SHOULD_ have to approve any tunnel being created at least once, IMHO.

> If the u/l bit is set to "unique" then they really are supposed to be globally unique... Obviously there's ample evidence of failures in this area, but how many affected devices are we talking abou…

“Supposed to be” and $10 will get you coffee at Starbucks.

In terms of the number of devices afflicted when you get down to consumer grade routers, I’d bet it might well be a few percent.

> There is no reason why one account couldn't set up multiple tunnels, so this shouldn't be an issue. The reason to have accounts at all is to be able to block them if people do bad things. Then you…

So why not just require a real account in all cases to begin with. It could easily be part of the router’s initial configuration wizard.

I simply don’t see the advantage to using a MAC address as an authentication mechanism when we can avoid all of those issues so easily.

> This is hard to avoid. Hopefully, with IPv6 there is enough tunneling that people will clean up their PMTUD breakage rather than let those with path MTUs below 1500 suffer, like what happens with I…

I’ve had <1500 MTU on both IPv4 and IPv6 for most of a decade. I ran into fairly regular problems in IPv4 until about 3 years ago. I’ve never seen significant problems in IPv6.

I realize this isn’t everyone’s experience, but I think for the most part unless you go specifically looking for brokenness or live in Japan, IPv6 PMTU discovery mostly works.

> A solution could be for home gateways that terminate a tunnel to advertise the tunnel MTU in their RAs so hosts use the tunnel MTU and also put that MTU in their TCP MSS.

I think if you just do the MSS thing, it’s usually adequate in my experience.

Owen

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 3 Nov 2014 23:11:41 +0000

> never seen significant problems in IPv6.

Maybe, but why settle for <1500 when you can have 1500+?

> > A solution could be for home gateways that terminate a tunnel to advertise the tunnel MTU in their RAs so hosts use the tunnel
> MTU and also put that MTU in their TCP MSS.
> 

Not all protocols include an MSS. Consider for example a tunnel that originates
from behind the home gateway. Recursively nested tunnels within tunnels
will be a fact of life, e.g., for anyone who sets up a VPN from a device on their
home network to their corporate office security gateway. And, links that
support  a degenerate MTU (i.e., anything less than 1500) always present
a limiting factor for recursion.

Thanks - Fred

---

From: Owen DeLong <[email-masked]>
Date: Mon, 3 Nov 2014 16:39:10 -0800

> On Nov 3, 2014, at 3:11 PM, Templin, Fred L <[email-masked]> wrote:
> 

I don’t see how you plan to achieve that with a tunnel. What I have is running code in production working on a variety of router platforms and my ISPs are willing to support it.

I know you’d like to see AERO replace all tunnels and more power to you, but as a datapoint for the discussion at hand, I don’t find that PMTU-D in the real world is as broken as some reports on here would have one believe.

> a limiting factor for recursion.

Yes, but they seem to mostly work with the ICMP PTB messages that come from the router in my experience.

If you fail to set the MSS on the router, you get packets that seem to fit within the MTU, but break TCP in interesting ways.

Owen

---

From: Jeroen Massar <[email-masked]>
Date: Tue, 04 Nov 2014 02:18:05 +0100

[ TL;DR: Please look at TSP and the TIC + (heartbeat or AYIYA) combo ]

On 2014-11-03 22:11, Owen DeLong wrote:
> So what you’re really saying here isn’t “MUST NOT be protocol 41” so
> much as “MUST BE use a known NAT-Friendly Transport Layer Protocol.”

I would not make that a "MUST".

I would make that 'requirement' a "should support NAT traversal".
The ability to 'upgrade' to proto-41 where possible is a good thing.
MTU-wise it is handy, as you can get to MTU = 1480 if lucky, but also
lots of systems do hardware-accelerated proto-41 processing, while that
won't happy quickly for a native protocol.

Then again.... as tests with sixxsd have shown, with a relative
'current' CPU, eg my iMac with a 3.4Ghz i7 (quad-core), one can easily
stuff a few gigabits of proto-41 and even AYIYA through it even though
all is userspace processed in that case.

The heavy part there is not the UDP actually, but the fact that AYIYA
does signature checking.

>> I use one every day so there's a lot right with them. The trouble
>> is that they involve too much configuration and aren't flexible
>> enough. For instance, when I get a new IPv4 address from my ISP I
>> need to go into the tunnelbroker.net web interface to update my
>> tunnel endpoint.
> 
> Sure, but a little bit of software added to your router using the API
> that HE publishes for that _COULD_ actually automate that.

You mean the 'standard' DynDNS API that is 'abused' for indicating a new
IP endpoint? Nice trick, and an easy way to get it supported in quite a
few devices.

The problem with that is that most of those tools do not have a notion
of what the 'current' IP is. And especially behind NATs, even 1:1 NATs
they won't notice an address change.

The problem with that again being that some random user that gets that
IP next will be getting proto-41 packets and will go "hey, you are
hacking" (we had the Dutch police on the phone because of such a
complaint in 2002 ;) But more importantly that it breaks connectivity
for a while.

Hence why a regular heartbeat is a good idea. Interface-notifications is
a separate step, but won't matter in the above 1:1 NAT example (thus
where proto-41 packets will pass).

AYIYA solves that by signing every packet, then even wifi/mobile roaming
just works as folks have been playing with their Android phones and
doing SIP calls for quite a while already.

>> But like I mentioned later in my message, I think much of what's
>> required can already be found in existing tunnel broker systems,
>> perhaps all that's needed is to bring it all together in a way that
>> allows it to be almost as easy to use as 6to4, while continuing to
>> benefit from the much better reliability of explicitly created
>> tunnels.

You cannot make it 'as easy to use as 6to4'.

Simply because to thwart abuse cases you need at minimum registration
with some kind of unique token that avoids re-signups.

The setups that have tried to play with anonymous connectivity tend to
end up firewalling everything left and right, typically long after
services have just blocked their networks for the amount of damage done
by the users of such service.

And as the latter is the case, the use of such a service by most people
just becomes useless as they won't be able to get anywhere. Sort of the
Tor effect: you anonimize, abuse gets made, you get blocked.

6to4 at least has the property that the IPv4 address of the real users
is embedded and thus can be blocked individually.

For tunnel broker systems, you just have to hope the provider enters
stuff in WHOIS properly and/or has a proper and working abuse@ address.

> Yep.
> 
> My point is that HE has actually already built all of the hooks
> necessary on their side. All that is needed is for router and/or host
> vendors to write the client-side glue.

Routers and vendors already implement TSP and TIC. Thus all the
components are in place.

Just requires HE to step up and finally implement these protocols that
have been out there for a long time.

I can help you with TIC if you want to :)

> Admittedly, that doesn’t work for NAT or CGN users because the HE
> tunnelbroker is protocol 41 based, but it does cover the vast
> majority of cases.

TIC can also just work for sole proto-41 if you want.

For dynamic addresses just add a simple heartbeat server and you are
done. Note that heartbeat signals the PoP directly, hence no need to
communicate these changes all the time from a central system that can
just fail, all PoPs are independent.

> Do you really expect vast numbers of CGN implementations that don’t
> provide native IPv6 at this point?

There are various ISPs who are doing the consumer-grade NAT thing for
years already.

The main motivation for the existence of AYIYA was Fastweb in Italy who
have been doing NAT for their users for well over 10 years now.

> Most of the implementations of CGN
> that I’m aware of are targeted at supporting IPv6 users that still
> need IPv4 access.

That is correct. They realize quite quickly that turning of or doing
something funny with IPv4 like translators causes lots of helpdesk calls
and thus ends up in big costs for them.

Hence, doing CGN is the best way to avoid most of it.

(Though lots of services completely break behind them... especially p2p
stuff, and that is just in the best 'interest' for most of those ISPS...)

[..context completely removed..]
> That requires some form of reliable way to make sure that all return
> traffic is routed back through the same gateway at the other side.

Lets not go down the 'anycast' rabbit hole again.

Teredo is still left doing that and the reliability/debugging problems
of that and 6to4 have been demonstrated well enough.

Hence, just let 1 IPv4 address talk to another 1 IPv4 address.

Exactly how currently Tunnel Brokers already work: nice and simple.

>> Another way to go would be to allow a few different types of
>> existing tunnel mechanisms, most notably 6rd.

6rd is for ISPs.

If an ISP choses to deploy 6rd they will get it to work. No changes
needed anywhere.

>> This would make the
>> system a little more complex on the client side, but allows for
>> reusing existing gateway side implementations.

As 6rd uses proto-41, indeed there are no changes needed, just some
deriving of tunnel endpoints based on the IP address.

>> If we go down that
>> route, it would be easy enough to include the option for the new
>> system to simply retrieve a fixed proto 41 configuration and be
>> fully compatible with existing tunnel brokers.

You mean, like TSP and TIC already do and how it is already implemented
in those client boxes? :)

>>> some development work on an HE tunnel.

Yes, please do implement TSP or TIC. You can chose between XML or simple
SMTP-style line-based commands.

I can help you with the latter if you want.

>> Tunnel down -> reinitiate it from scratch.

Why 'reinitiate'? This is what TSP does, why bother with losing those
packets?

>> Simple enough for the
>> client. The tunnel broker would need some way to get the new client
>> address, though.

heartbeat + AYIYA (which has per-packet heartbeats) solve this just fine.

> 1.	These are intended to be stateless tunnels, so authentication
> isn’t feasible because authenticated vs. unauthenticated _IS_ state. 

Indeed, you need at least the 'state' of the 'user IPv4 endpoint', next
to the 'configuration parameters' of 'IPv6 prefix', 'MTU' and
'password', nothing much else is needed though.

> 2.	The client can easily know when its address has changed.

As per the above, not in all scenarios. But yes, it can try to figure it
out with interface-change events and similar setups.

> OTOH,
> other than by address, it’s hard for the tunnel broker to know, and
> if the address no arriving is unknown, virtually impossible to know
> what tunnel to associate it with and update.

Exactly. Hence why those packets get a proto-41 unreachable.

> Currently, using existing APIs, the client, upon detecting a new
> address could access and reconfigure the Tunnel Broker and the tunnel
> would come back up.

Or it could just send a simple heartbeat directly to the PoP and voila
you got something people have been using for 10+ years already :)

> I had, at one point, scripted most of this on my
> site, but it was fragile because the script depended on polling my
> router from a third host and then connecting to the tunnel broker
> OOB.

Why do that? Just put an identity in a packet, add a current timestamp
(against replay attacks :) and hash that with a password.

Very simply, very quick and lean, implementable on basically everything.

And run that next to your proto-41 tunnel: it is called heartbeat!

If you need to cross NATs: use AYIYA ;)

>>> Since all packet forwarding requires some level of ISP
>>> cooperation, I think you need to be a little more clear about
>>> your meaning here.
>> 
> Sure, on this I agree (presuming that simply forwarding packets is
> within the bounds of not doing anything to help), but yes, I agree in
> the context of “ISP shouldn’t have to host or participate in
> configuring or provisioning tunnels or providing tunnel termination
> services in order for it to work”.

Like every Tunnel Broker in the world has been doing for years already;
except that some think that proto-41 passes to every box in the world...

>> It still working if the ISP goes out of its way to filter would be
>> another step. And I don't think we should attempt that, as it would
>> be hard to do and also because non-ISP networks such as businesses
>> and universities have a reasonably legitimate reason to not want to
>> have arbitrary tunnels on their networks.
> 
> Agreed.
> 

That is called 'vote with your money'. People will vote, especially when
they notice how thing the pipes and hardware really are that their
packets are flowing through...

>> Obviously we'd want those networks to offer native IPv6, but if our
>> new tunnel mechanism can be terminated on the inside of such
>> networks and then firewalled the same way as other traffic would
>> also be a good solution.
> 
> Ideally, yes, but doing this (firewalling tunneled traffic) in an
> IPv6-ignorant network is difficult at best.

Depends on who wants to do the firewalling, if it is the ISP, take the
AT&T example, they filter proto-41 but forget all the other VPN
protocols out there.

If it is the user, they know how iptables/WindowFirewall/etc works right?

>>>> - traffic flows are reasonably optimized (i.e., if ISP offers
>>>> gateway service, by default, that gateway is used)
>> 
>>> This requires some form of service discovery mechanism that can
>>> somehow be aware of what ISP you are using or otherwise has some
>>> mechanism for automatically interfacing with said ISPs
>>> provisioning.
>> 
>> Yes.
>> 
>>> In the tradition of rough consensus and running code, it would be
>>> great if you put forward your vision for how this would work.
>> 
>> What I have in mind is that there is a list of gateways and users
>> are connected to the gateway that serves them best, which would
>> typically be their ISP's gateway if they run one.
> 
> How would this list be maintained? What would be the bar for getting
> on the list? What would be the mechanism for maintaining the list?
> How would a device locate the list at bootstrap?

Such a list does not work indeed, not an automatically parseable one as
Tunnel Brokers have different methods anyway.

AICCU still has a feature where we stuck TXT records with details of
various brokers in DNS under _aiccu.sixxs.net. But most of those TBs
went away and we thus emptied the list.

Screenshot of the Debian edition of that still available here:
http://podpora.nic.cz/files/podpora/ipv6/aiccu_installation.png

Nope, Hurricane Electric was never in that list as they still do not do
either TSP or TIC and without an automatic way to fetch the config,
little sense to put them in there...

Google is very quick to reveal the following though:

https://en.wikipedia.org/wiki/List_of_IPv6_tunnel_brokers

People can go from there.

Also, people who cannot find a Tunnel Broker, well, IMHO they do not
need IPv6 yet.

> This is a nice hand wave and I understand the general desired
> outcome, but there are many devils in the details.
> 

Lots of the code discussed in this thread has been running for 10+ years
and is actively deployed on a lot of platforms.

Join the fun I would say ;)

>>> I definitely think that deprecation of configured 6in4 tunnels
>>> would be very premature and somewhat harmful in general.
>> 
>> But do you agree that the current tunnel broker system is too
>> complex to serve the role that 6to4 was intended to serve?
> 
> I’ve never really been completely clear on the intended role for 6to4
> since it so clearly missed whatever mark that was, so it’s hard to
> comment authoritatively.

The intended role of 6to4 was to automatically give every IPv4 endpoint
(at least publicly routed ones) a IPv6 address + /48.

It worked like a charm for quite a while when one was selecting the
relay by hand as then you effectively had a tunnel broker setup.

Then the anycast node got added and some ISPs setup an instance which
was great. But then people started shoving lots of crap over it and
especially those instances where some German ISP found it 'okay' for
their customer to send a gigabit of spoofed 6to4 packets caused several
anycast nodes to shutdown due to traffic overload which caused a domino
effect and suddenly there where not many of those nodes left.

> I will say that in my experience, no, most of the tunnel broker
> systems are not that hard to use. What they lack is sufficient
> automation to achieve the self-configuring aspects of 6to4. IMHO,
> said automation could be built onto the existing services without too
> much effort. What I don’t get is why none of the router vendors that
> set out to do this have followed through.

I think you should read the rest of this thread, lots of vendors support
TSP and TIC out of the box...

The problem with Tunnel Brokers then becomes getting an account and not
lying at the signup or providing broken data though.

>>>> The router or host, when it notices that there is no native
>>>> IPv6, connects to _tunserv._tcp.<domain> or something like that
>>>> and authenticates. The setup server then does a lookup of the
>>>> source IPv4 address and redirects to a gateway...

Which 'domain' would be used for such a setup?

I've suggested this a long time ago:
 http://jeroen.massar.ch/archive/drafts/draft-massar-v6ops-tunneldiscovery-00.html

But the discussion then concluded that it was not going anywhere as
there is no way to discover them.

ISPs that are not cooperating for getting IPv6 deployed won't have the
record, thus leaves systems like Tunnel Brokers where the label will be
easy for the user: for TIC: tic.sixxs.net (or tic.<isp>) or for TSP
whatever the host of the Gateway6 box is.

>>> This assumes a tremendous amount of cooperation and coordination
>>> not in evidence. Who runs this central "setup server" service?
>>> Why? How is it paid for?

SixXS is just done for the fun of it. But yes, that costs a lot of time,
and the biggest problem of that: can't help every user in the world, a
day only got 24 hours...

>> What I imagine is that existing cloud services would incorporate
>> that, most notably Apple, Google and Microsoft, who all have shown
>> various levels of interest in getting IPv6 working well.

Google didn't want to go for it the last time we discussed it with them.
Their biggest problem is "state". Even though there are few state
variables scaling it up to Google sizes where the world can actually use
it will not scale as then you need to distribute that state and do that
near-instant as we are talking about packets.

>> But also
>> any other interested party, such as the existing tunnel brokers.

>From my SixXS POV, there is nothing to be done, as we already have
provided the solutions to the problems that have been listed in this
thread: TIC, heartbeat & AYIYA.

>> The technical part is basically running a web server and a user
>> database, which doesn't cost a meaningful amount of money.

And where are you going to move your packets through? :)

> That can cost anywhere from virtually nothing to very much.

Exactly. Except for a LOT of sparetime, SixXS has not cost too much for
that matter; everything costing actual money has been donated. (nope, I
won't calculate based on an hourly rate how much the time involved cost)

Though, that is why keeping the 'fun' in it important, the moment that
is gone, and there is nothing to learn anymore it will become painful.

>> The harder part is the coordination, but the benefit is huge: you
>> get a tunnel that's terminated as close to the user as possible
>> without requiring any action from the user to accomplish this.
> 

I fully agree.

Yep.

Especially as then the user is aware that this new IPv6 thing punctures
a hole in all the IPv4 firewall policies that they previously
meticulously closed up.

[..]
>> There is no reason why one account couldn't set up multiple
>> tunnels, so this shouldn't be an issue. The reason to have accounts
>> at all is to be able to block them if people do bad things. Then
>> you are blocked if you use the default account and the user of a
>> clone of your MAC address does bad things. So set up a real account
>> and your tunnel is back.
> 
> So why not just require a real account in all cases to begin with. It
> could easily be part of the router’s initial configuration wizard.
> 
> I simply don’t see the advantage to using a MAC address as an
> authentication mechanism when we can avoid all of those issues so
> easily.

I agree. Just let people sign up. This also makes the Tunnel Broker the
point where they need to get their service and agree to their terms.

No change here thus.

>>> Also, ip-proto-41 does have path MTU challenges. We all know that
>>> PMTUD is unreliable, which means the tunnel ingress has to set a
>>> "safe" MTU - usually to something on the order of 1480 or
>>> smaller.
>> 
>> This is hard to avoid. Hopefully, with IPv6 there is enough
>> tunneling that people will clean up their PMTUD breakage rather
>> than let those with path MTUs below 1500 suffer, like what happens
>> with IPv4.
> 
> I’ve had <1500 MTU on both IPv4 and IPv6 for most of a decade. I ran
> into fairly regular problems in IPv4 until about 3 years ago. I’ve
> never seen significant problems in IPv6.

Unless your name is Akamai and have random nodes b0rking:

https://www.sixxs.net/forum/?msg=general-12378937

> I realize this isn’t everyone’s experience, but I think for the most
> part unless you go specifically looking for brokenness or live in
> Japan, IPv6 PMTU discovery mostly works.

IPv6 PMTU works fine. It is broken networks that do not work fine.

In the Akamai case it likely is some kind of software bug, as they know
about these issues for a few years already and they have good folks.

Typically though you find IPv6 PMTU issues in Enterprisy Networks where
some consultant went and said "I am an IPv6 Guru, you do not need that
ICMP stuff, just like IPv4".

>> A solution could be for home gateways that terminate a tunnel to
>> advertise the tunnel MTU in their RAs so hosts use the tunnel MTU
>> and also put that MTU in their TCP MSS.
> 

Clamping MSS is *avoiding* the issue.

If you are clamping MSS then you are indicating that you have MTU issues.

And you will only notice that when you start using UDP based services
again...

Greets,
 Jeroen

---

From: Keith Moore <[email-masked]>
Date: Mon, 03 Nov 2014 21:55:43 -0500

> I agree somewhat. But the real question is how much longer it's going to take rather than how long it's been under way already... Native IPv6 deployment is picking up speed, but on the other hand e…

Even if deployment gets to say 60-70% within a couple of years, I see a 
good chance of there being a fairly long "tail" of IPv6 deployment.

Keith

---

From: Owen DeLong <[email-masked]>
Date: Mon, 3 Nov 2014 19:14:21 -0800

> IP endpoint? Nice trick, and an easy way to get it supported in quite a

No, I meant the HE HTTP based API for manipulating tunnel broker directly.

I admit it’s not well known, but it does exist.

> they won't notice an address change.

That’s pretty easy to detect.

I admit I haven’t done anything that attempts to do tunnels dynamically, but I find the concept of a home router that doesn’t known what its IP address is a bit strange. I can understand the CGN case where the router doesn’t get a public address on the outside, but even there, if you know your “external” address isn’t, it’s not hard to have logic like:

	if (packet_not_received_thru_tunnel_in_last_30_seconds)
	{
		replied = ping(remote_tunneled_interface_address)
		if(!replied)
		{
			reconfigure_tunnel(find_current_address())
		}
	}

If you don’t like 30 seconds, pick a value you’re comfortable with.

> complaint in 2002 ;) But more importantly that it breaks connectivity

That’s funny, but I’m thinking it didn’t take too long to teach the reality to the police.

See above, I don’t think it has to break connectivity for long enough to matter all that much.

> where proto-41 packets will pass).

Regular heartbeat doesn’t have to be built into the protocol. There’s no regular heartbeat in PPP, but thousands (if not millions) of BGP sessions successfully fall over every day due to TCP timeouts when PPP stops working on those links.

> doing SIP calls for quite a while already.

Tradeoff is more overhead in your packets. I’m not opposed to AYIYA or whatever works, but I still don’t think we need a new protocol or any real IETF work to achieve your goals.

> You cannot make it 'as easy to use as 6to4’.

Why not?

> with some kind of unique token that avoids re-signups.

6to4 does nothing to thwart abuse cases, so you’ve added a requirement not present in the 6to4 system to begin with.

In order to do that, I think the best thing is user registration with appropriate PII. Make sure that you have a place to send the LEOs if they abuse the system.

> by the users of such service.

I’m well aware of this. (Though interestingly, 6to4 seems not to suffer from
this problem somehow).

> Tor effect: you anonimize, abuse gets made, you get blocked.

Which is why a registered account makes more sense to me. However, that’s just an additional half-page form in the setup wizard on the box. No, it’s not as easy as 6to4, but neither is anything else that isn’t anonymous on some level.

> is embedded and thus can be blocked individually.

Well, you could create a mapping and open up lookups along the same lines…

	Each unique IP that has a registered tunnel is mapped to a 16-bit number.
	The same 16 bits serves as the network number (allocate a /32 to the tunnel broker)
	If you need to support more than 65,536 tunnelbroker users, get another /32.

	Then, provide a map where people can query a /48 prefix number and get back
	the real IPv4 address.

	Now you have effectively the same capabilities as 6to4 in terms of abuse prevention

> stuff in WHOIS properly and/or has a proper and working abuse@ address.

More accurately, if you’re providing tunnel broker services, it’s important that you
be responsive to abuse complaints and have some way to track abuse back to
the original user.

HE does that all the time.

I presume the other reputable tunnel brokers (SIXXS, GOGO) do as well.

> have been out there for a long time.

I don’t think that’s likely.

> I can help you with TIC if you want to :)

Apparently you haven’t heard… I’m no longer with HE. I still use an HE tunnel
for some of my IPv6 upstream connectivity, but I’m an ordinary user like anyone
else.

> just fail, all PoPs are independent.

There are a number of reasons I don’t think that’s always desirable from the
perspective of the tunnel broker provider.

> have been doing NAT for their users for well over 10 years now.

That’s just sad. They aren’t doing IPv6?

> Hence, doing CGN is the best way to avoid most of it.

My point is that if they are doing native IPv6 + CGN/v4, then their users don’t
really need a 6over4 mechanism or tunnel broker. What am I missing?

> stuff, and that is just in the best 'interest' for most of those ISPS…)

Don’t get me started, but it’s not really the topic of this thread.

> Lets not go down the 'anycast' rabbit hole again.

I’m not talking about any cast, I’m talking about truly stateless routers and the potential of more than one tunnel serving the same end user.

> of that and 6to4 have been demonstrated well enough.

Yep… I wasn’t disagreeing with this.

> Hence, just let 1 IPv4 address talk to another 1 IPv4 address.

Agreed, but what you’re talking about would require the ingress traffic to always traverse the same tunnel as the egress traffic. I’m calling that broken.

> Exactly how currently Tunnel Brokers already work: nice and simple.

I can send a packet out through my HE tunnel and get the answer back through my tunnels to Layer 42 and everything is just fine. In your model, I believe that would have broken.

> 'password', nothing much else is needed though.

Those are configuration parameters, not state. If you are not maintaining
some form of state for authenticated vs. unauthenticated (persistent TCP
session, for example), then you have to carry an authentication token in
every packet. Since one of the complaints here is MTU reduction, it seems
to me that adding overhead to the packet is contrary to the desired result.

Hence, eliminating the authenticated vs. unauthenticated state seems
desirable. Instead, provide a stateful authenticated way to configure
“end-point address” out of band from the tunnel and let the tunnel
be stateless thereafter.

> out with interface-change events and similar setups.

Again, I consider CGN clients without IPv6 connectivity to be more of
a corner case than a core solution space.

> Exactly. Hence why those packets get a proto-41 unreachable.

Which should tell the client it’s time to renegotiate its address if it hasn’t
figured it out yet.

> you got something people have been using for 10+ years already :)

Again, at the price of MTU reduction and a stateful tunnel.

> (against replay attacks :) and hash that with a password.

Because I don’t control the software at the other end and this was a solution
I could apply on the client side of the tunnel.

> Very simply, very quick and lean, implementable on basically everything.

Virtually anything you can think of is “implementable on basically everything”
if you’re willing to apply enough resources to the problem. I think someone
once ported an IPv4 stack to a Commodore Pet. It took 47 of the 48k of RAM
in the machine, so you couldn’t actually _DO_ anything more sophisticated
than TELNET with it, but it was “implemented”.

However, the solution you propose requires modifying software on the server
side as well as the client in a coordinated manner. OTOH, what I proposed
could be implemented 100% client side which has significant advantages
when it comes to adoption in an environment where the client is motivated
and the tunnel broker is providing a free service to the community.

> And run that next to your proto-41 tunnel: it is called heartbeat!

I have not had the best experiences with OOB heartbeats. YMMV.

> except that some think that proto-41 passes to every box in the world…

Not every box in the world, just every box on the actual internet.

> packets are flowing through…

Problem is there are many areas in the US where you get your residential
internet from AT&T or you don’t get residential internet.

Oligopolies and monopolies make it very difficult to vote with your money
in a meaningful way.

FWIW, I have voted with my feet and do not subscribe to any AT&T residential
services any more.

> AT&T example, they filter proto-41 but forget all the other VPN

Ah… Here’s the worst of the AT&T situation…

They aren’t doing it for security or for any sort of protocol inspection or such.

They blocked external protocol 41 for ALL of their customers when they started
turning up 6rd for SOME of their customers because they don’t want to risk
certain forms of customers circumventing their 6rd service.

> If it is the user, they know how iptables/WindowFirewall/etc works right?

ROFLMAO… Generally, no. At least not in my experience.

> Also, people who cannot find a Tunnel Broker, well, IMHO they do not

But you’ve once again left the realm of “as easy to use as 6to4”.

Make up your mind. You are vacillating  wildly between “easy to use” and
“you must be this tall to ride” and I’m getting whiplash trying to keep up.

> Join the fun I would say ;)

My point all along has been that there’s running code to do what you want
(or at least come very close to it). There’s nothing for the IETF to do here.
There’s some middleware that is needed/missing.

> effect and suddenly there where not many of those nodes left.

TTBOMK, the HE and Comcast nodes are still up and running.

> day only got 24 hours…

Right… But if you want router vendors to start putting this in their boxes,
you probably need something with more assurance that it will keep going
for at least some reasonable amount of time.

Don’t get me wrong, I think SIXXS is doing cool stuff and doing it well.
I think they will be around for quite a while. But I wouldn’t be able to tell
a product marketing manager to depend on it with a straight face.

> is gone, and there is nothing to learn anymore it will become painful.

Yep. Further, the moment it becomes painful, it will implode.

> IPv6 PMTU works fine. It is broken networks that do not work fine.

Now we’re dividing rabbits unnecessarily.

I meant from a real-world user perspective, IPv6 PMTU discovery works in almost
every circumstance and I have encountered few reachability problems as a result.

Other than people who go specifically looking for broken networks (a worth while
endeavor) or people subject to the various intricacies of IPv6 implementations in
Japan, there aren’t that many broken networks out there from an IPv6 PMTU-D
perspective.

> If you are clamping MSS then you are indicating that you have MTU issues.

On my Juniper, if I didn’t clamp MSS to an appropriate value to go with the tunnel MTU, then non-TCP packets worked fine, but TCP sessions would hang. When I clamped the MSS value, it resolved the TCP problems. I can now reach TCP, UDP, and ICMP services with packets larger than 1500 in both directions with no issues.

For some reason, changing the interface MTU on a Juniper doesn’t seem to clamp the MSS the way one would expect it to, so that has to be done manually.

(At least in the release of JunOS that I am running).

> again…

I use many UDP based services every day.

Perhaps, instead of assuming that you know better than everyone else, read what they write and consider the full context.

I don’t change the MTU in my RAs because I have a full 1500 octet MTU on all my LAN segments and that works just fine. I do have a smaller MTU on my tunnel interfaces because they have to be encapsulated and the encapsulated packets then traverse networks with 1500 octet MTUs.

The TCP MSS configuration value is box-wide and there are two places where it might need to be set.

system->internet-options->tcp-mss
and
security->flow->tcp-mss

Owen

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 4 Nov 2014 16:09:29 +0000

Hi Owen,

> Sent: Monday, November 03, 2014 4:39 PM
> To: Templin, Fred L
> Cc: Iljitsch van Beijnum; [email-masked] WG
> Subject: Re: Bar BoF on a 6to4 replacement? Was: my recommendation re: 6to4
> 
> 
> >> I’ve had <1500 MTU on both IPv4 and IPv6 for most of a decade. I ran into fairly regular problems in IPv4 until about 3 years ago.
> I’ve
> I don’t see how you plan to achieve that with a tunnel.

It is in Section 3.12 of 'draft-templin-aerolink'.

> What I have is running code in production working on a variety of router
> platforms and my ISPs are willing to support it.
> 
> I know you’d like to see AERO replace all tunnels and more power to you, but as a datapoint for the discussion at hand, I don’t find
> that PMTU-D in the real world is as broken as some reports on here would have one believe.

We have debated that heavily over the last decade, and there is a raft of
RFCs that document the problems. Essentially, IPv4 fragmentation is bad,
and ICMP PTBs can be dropped and/or forged by adversaries. So, an
approach that avoids the problems while offering an assured 1500 might
be useful. And, if packets larger than 1500 can make it through the tunnel
we let them through too.

This latter point is worth highlighting. By setting a fixed <1500 MTU on a
tunnel interface as per the current practices you are describing, we will
never get to a jumbo-capable Internet. By allowing 1500+, we might
just stand a chance.

Thanks - Fred

> >> I realize this isn’t everyone’s experience, but I think for the most part unless you go specifically looking for brokenness or live in
> Japan,

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 12 Nov 2014 10:12:17 -1000

I've been talking to a few people and I think there is potential for a draft that lets home gateways (and any other system so inclined) have IPv6 tunneling enabled out of the box (like 6to4) but in a way that works pretty reliably (unlike 6to4).

My question to (potential) implementers:

- Is requiring HTTP digest authentication problematic?
- Is requiring HTTPS problematic?
- Is requiring JSON parsing problematic?
   * Would using plain text similar to HTTP headers to pass parameters be better?
- Is requiring DHCPv6 (the prefix delegation client role) problematic?

Thanks,

Iljitsch

---

From: Jeroen Massar <[email-masked]>
Date: Wed, 12 Nov 2014 21:38:04 +0100

On 2014-11-12 21:12, Iljitsch van Beijnum wrote:
> I've been talking to a few people and I think there is potential for
> a draft that lets home gateways (and any other system so inclined)
> have IPv6 tunneling enabled out of the box (like 6to4) but in a way
> that works pretty reliably (unlike 6to4).

"pretty reliably" or "reliable"

As somebody who is terminating an extreme amount of tunnels to get those
people who don't have native IPv6 some IPv6: can you provide more
details please of what direction you are thinking of?

What does 6rd not resolve and what does this better?

And as below you are asking for HTTP things, what is it that TSP and TIC
do not resolve or do 'incorrectly'?

Note that there is no magic "IPv6 Tunnel Broker Discovery". Somebody has
to type a URL somewhere or provide parameters.

And the moment that you can do DHCPv4 you can do 6rd.

> - Is requiring JSON parsing problematic?

Typically these components are already available in those systems.

The problem with HTTPS and anything security though is this annoying
thing called time. Most CPEs are not NTP synced as they do not have a
RTC. And without proper time you can't do proper TLS or any other
variant of crypto that has timestamps for replayattack protection etc.

>   pass parameters be better?

Depends completely on what you are talking about. Examples would be useful.

> - Is requiring DHCPv6 (the prefix delegation client role)

Any proper "IPv6 capable CPE" SHOULD have this.

Greets,
 Jeroen

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 12 Nov 2014 11:06:05 -1000

> "pretty reliably" or "reliable"

Nothing is perfect in this world, but the idea is that it should be so reliable that it can safely be turned on "set and forget" by default.

> details please of what direction you are thinking of?

Sure.

> What does 6rd not resolve and what does this better?

6rd requires cooperation from your ISP. Now if your ISP offers 6rd, you should use it. But a third-party home gateway doesn't necessarily know how to configure itself to use 6rd for your ISP. If your ISP doesn't do 6rd but you have a SixXS account, you'll want to use that account and set up an AYIYA tunnel - but only if your home gateway supports AYIYA. But maybe you have a public IPv4 address so you could use a proto 41 tunnel.

Today, determining all of this, selecting the right tunnel and then configuring at least some parameters are done manually. Which is a non-starter for most users. The draft aims not so much to come up with a new tunneling mechanism but rather to allow devices to find and configure existing tunneling options.

However, I do think it will be necessary to define an additional tunneling option that's simple to implement that could be the "mandatory to implement" default one.

> do not resolve or do 'incorrectly'?

The main issue is that you need to talk to someone who runs TSP or TIC. So this wouldn't address the case where an ISP runs 6rd. Or you have an account at a tunnel broker that doesn't use TSP or TIC (yes, I know, hard to imagine).

> Note that there is no magic "IPv6 Tunnel Broker Discovery".

So let's invent it!

> Depends completely on what you are talking about. Examples would be useful.

Login: [email-masked]
Password: ALLYOURBASE64AREBELONGTOUS
Encap-Supported: proto41, rfc5969, net.sixxs.ayiya
Negotiation-Supported: DHCPv6-PD, DHCPv6-IA, rfc4862, net.sixxs.tic
Client-Address: 100.64.0.1
Client-Type: router

---

From: Gert Doering <[email-masked]>
Date: Wed, 12 Nov 2014 22:33:33 +0100

Hi,

On Wed, Nov 12, 2014 at 10:12:17AM -1000, Iljitsch van Beijnum wrote:
> I've been talking to a few people and I think there is potential
> for a draft that lets home gateways (and any other system so inclined)

I'm not sure where the value of that is.

10 years ago, I would have agreed that "getting IPv6 out to the people"
is a good goal - but when this draft gets to the stage that you'll see
devices actually shipping, it's 2016, and we'll have 30-40% native IPv6 
in many countries already - which should create enough momentum by it's
own...

So, call me sceptic :)

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: Tim Chown <[email-masked]>
Date: Wed, 12 Nov 2014 22:22:16 +0000

> On 12 Nov 2014, at 21:33, Gert Doering <[email-masked]> wrote:
> 
> Hi,
> 

Indeed.

Tim

---

From: Jeroen Massar <[email-masked]>
Date: Wed, 12 Nov 2014 23:25:11 +0100

Btw +1 on Gert Doering's comment "when this draft gets to the stage that
you'll see devices actually shipping, it's 2016, and we'll have 30-40%
native IPv6". I see it the same way, by then IPv6 should be much better
spread. The people who want IPv6 will get it using the available
methods, no need to standardize further; the testing base is large
enough for real deployments to move on. Folks who just want "internet"
(typically the HTTP/S portion) won't care about it (IPv4 or IPv6) anyway.

[..]
> 6rd requires cooperation from your ISP. Now if your ISP offers 6rd,
> you should use it. But a third-party home gateway doesn't necessarily
> know how to configure itself to use 6rd for your ISP. 

In all cases that I am aware of where ISPs chose to use 6rd those ISPs
provide the CPEs or upgraded the CPEs to support 6rd.

Though, for instance Swisscom.ch made the details of their 6rd relay
available before that so that people could also test it with other CPEs.

>If your ISP
> doesn't do 6rd but you have a SixXS account, you'll want to use that
> account and set up an AYIYA tunnel - but only if your home gateway
> supports AYIYA. But maybe you have a public IPv4 address so you could
> use a proto 41 tunnel.
> 
> Today, determining all of this, selecting the right tunnel and then
> configuring at least some parameters are done manually.

As one really nice example:
https://www.sixxs.net/wiki/Fritz!Box_general

- User creates an account (most difficult part)
- Request a heartbeat tunnel
- Fill in username/password + tunnel_id in Fritz!Box
- press "go".

Though, that is the most simple of them. Other vendors have similar
configuration options.

> Which is a
> non-starter for most users. The draft aims not so much to come up
> with a new tunneling mechanism but rather to allow devices to find
> and configure existing tunneling options.

Discovery is an over hard thing. Especially as the market provides a
variety of providers and such a mechanism requires centralization which
none of them will like and nobody will properly maintain.

Currently most folks will check:
https://en.wikipedia.org/wiki/List_of_IPv6_tunnel_brokers
or do a Google^WBing search.

and based on the properties of the providers, who they like and not,
just like the normal DSL/Cable market, select the provider of choice.

Indeed, folks who do not care about IPv6 won't do that. They are not
missing out, and will get IPv6 one day when their normal provider turns
it on.

> However, I do think it will be necessary to define an additional
> tunneling option that's simple to implement that could be the
> "mandatory to implement" default one.

That would be protocol-41. Afaik every Tunnel Broker does that. (though
with Gogo6/Freenet6 I recall it is a fallback in TSP)

TSP or AYIYA would be reasonable (IMHO) alternatives.

Note that even Fritz!Box does not do AYIYA, then again it is mostly
configured to sit on the public IP address (till the providers stuffs
you behind CGN and that breaks).

Currently to change protocols in SixXS between proto-41/heartbeat/AYIYA
one has to change it in the UI; I have plans for making it possible to
switch those based on incoming (signed) packets though, but something
with available time.

>> And as below you are asking for HTTP things, what is it that TSP
>> and TIC do not resolve or do 'incorrectly'?
> 
> The main issue is that you need to talk to someone who runs TSP or
> TIC.

When a provider supports it you'll just have to select that option and
provider the name of the server (next to username/pass/tunnel_id).

Hence, no need to do an extra protocol there.

> So this wouldn't address the case where an ISP runs 6rd.

6rd details get distributed by DHCPv4. No need to solve that.
See http://tools.ietf.org/html/rfc5969#section-7.1.1

> Or you
> have an account at a tunnel broker that doesn't use TSP or TIC (yes,
> I know, hard to imagine).

Not hard to imagine: Hurricane Electric does not provide either.

All their users manually cut & paste the parameters into their devices.

Which btw is an option in SixXS since the IPng.nl days; though it won't
easily work for heartbeat/AYIYA that have a password for signing packets.

>> Note that there is no magic "IPv6 Tunnel Broker Discovery".

There have been discussions about this in the past and the conclusion
was that a centralized service is unmaintainable and other options just
rely on DNS attributes that would have to be either configured or that
the user would have to indicate the label to use; which is just akin to
server selection.

> Login: [email-masked]

That line @homegateways.org means that you have a centralized registrar.
That list will never be up to date, nor will anybody like it that
something like that is out of their control, eg because of outages which
would thus cause their service to fail.

The portion behind the @ could just be the server provider, and voila,
you are selecting the negotiation protocol, be that TIC or TSP.

> Negotiation-Supported: DHCPv6-PD, DHCPv6-IA, rfc4862, net.sixxs.tic

If you have DHCPv6 the local provider is already handling you and you do
not care about tunnels.

If you have TIC support, well, the user can change that "TIC Server"
option quite easily (though most UIs don't expose it ;) as then you know
what the TIC server address is.

Same for TSP, there you just fill in the TSP server name +
username/password and done.

> Client-Address: 100.64.0.1

The client never knows it's address; especially when it is behind a CGN
or multiple layers of that nonsense.

> Client-Type: router

In IPv6 everything is considered a router. Most Tunnel Brokers will per
default route a /64 to the tunnel endpoint.

In SixXS that is simply possible as it is the tunnel address with the
49th bit set to 1 (hence why one sees an '8' there) and as it is that
simple, it is a direct jump table, hence no lookups required and thus
very fast. (Before sixxsd it would have required an extra /64 entry in
the kernel and the typical Free OS requires a big prefix tree for
looking up 16k routes; while with this trick it is there directly ;)

Greets,
 Jeroen

---

From: Alexandru Petrescu <[email-masked]>
Date: Thu, 13 Nov 2014 06:09:31 +0100

> - Is requiring HTTP digest authentication problematic?

Something I can foresee as a problem is sharing the keys, or do any 
encrypted or authenticated exchange, with someone who is on the other 
side of the globe, someone non-accountable in any particular way.  A 
matter of trust.

> - Is requiring DHCPv6 (the prefix delegation client role) problematic?

I guess it is problematic.  I am not aware of any operator of DSL or 
cellular links that offers DHCPv6 Prefix Delegation service (a Server 
that replies positively to requests for prefixes).

On another hand, if we talk DHCPv6 Prefix Delegation over a tunnel (like 
running it over the tunnel already established by e.g. HE), then there 
should be no problem in using it.

FWIW.

Alex

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Thu, 13 Nov 2014 07:09:37 +0000

> I guess it is problematic.  I am not aware of any operator of DSL or cellular
> links that offers DHCPv6 Prefix Delegation service (a Server that replies
> positively to requests for prefixes).

I am absolutely aware of at least one DSL operator who does this.
It's definitely a part of the Broadband Forum architecture that many such operators helped write.
http://www.broadband-forum.org/technical/download/TR-177.pdf 
There exists no native IPv6 architecture from BBF that does not make use of DHCPv6 IA_PD.
Barbara

---

From: Doug Barton <[email-masked]>
Date: Wed, 12 Nov 2014 23:45:47 -0800

> in many countries already - which should create enough momentum by it's

+1

We don't need this, shouldn't develop it, and definitely shouldn't 
deploy it.

Doug

---

From: Gert Doering <[email-masked]>
Date: Thu, 13 Nov 2014 09:15:49 +0100

Hi,

> that replies positively to requests for prefixes).

DHCPv6-PD is the standard method to hand out prefixes on DSL lines here
in DE.  ("What else could you use?").

So, now you are aware of DSL operators that do this, like "Deutsche Telekom".

Gert Doering
        -- NetMaster

---

From: Doug Barton <[email-masked]>
Date: Thu, 13 Nov 2014 00:30:54 -0800

> So, now you are aware of DSL operators that do this, like "Deutsche Telekom".

FWIW, support for this is also built into OpenWRT.

Doug

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 13 Nov 2014 09:21:42 -1000

> Something I can foresee as a problem is sharing the keys, or do any encrypted or authenticated exchange, with someone who is on the other side of the globe, someone non-accountable in any particular way.  A matter of trust.

Yes, it would be somewhat risky to put the cloud credentials that give access to your entire digital life inside a $50 home gateway. So it would be good to require OAuth (RFC 6749). On the other hand, that could be quite a barrier for implementers.

However, I'd say that HTTP digest authentication is secure enough to protect an account that is only used to set up a tunnel.

> I guess it is problematic.  I am not aware of any operator of DSL or cellular links that offers DHCPv6 Prefix Delegation service (a Server that replies positively to requests for prefixes).

It doesn't matter what the network operators do. If they provide native IPv6, then the whole effort is moot. What counts is what we can reasonably ask home gateway vendors to implement.

Iljitsch

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 13 Nov 2014 20:34:16 +0100 (CET)

On Thu, 13 Nov 2014, Iljitsch van Beijnum wrote:

> It doesn't matter what the network operators do. If they provide native 
> IPv6, then the whole effort is moot. What counts is what we can 
> reasonably ask home gateway vendors to implement.

I am of the opinion that we shouldn't have any more "let's create some 
automatic IPv6 tunneling thingie that might work". 6to4 was nice to get a 
way for people to try out IPv6. There are now tens of millions of 
households with native IPv6 connectivity in the world without much ill 
effects. It's now up to the operators to roll out to the rest of the mass 
market with proper reliability and quality, and this is happening.

I would rather people had good IPv4 connectivity and no IPv6 connectivity, 
than them having good IPv4 connectivity, so-so IPv6 connectivity, people 
learning that and turning off IPv6 to fix their problem, and then when 
their operator rolls out proper IPv6, the end user system still has IPv6 
turned off.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Nick Hilliard <[email-masked]>
Date: Thu, 13 Nov 2014 19:39:01 +0000

> their operator rolls out proper IPv6, the end user system still has IPv6

Part of that problem is making ipv6 simple enough for CPE vendors to create
products which are reliable enough to stop eyeballs from thinking that
disabling ipv6 will be a net gain for them.  Creating more protocol soup
will not help with this aim.

Nick

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 13 Nov 2014 09:50:31 -1000

>> It doesn't matter what the network operators do. If they provide native IPv6, then the whole effort is moot. What counts is what we can reasonably ask home gateway vendors to implement.

(For the record, many DSL operators that provide native IPv6 do support DHCPv6-PD.)

> I am of the opinion that we shouldn't have any more "let's create some automatic IPv6 tunneling thingie that might work". 6to4 was nice to get a way for people to try out IPv6. There are now tens of millions of households with native IPv6 connectivity in the world without much ill effects. It's now up to the operators to roll out to the rest of the mass market with proper reliability and quality, and this is happening.

I wouldn't declare victory quite yet. According to Google we're now pushing 5% (native) IPv6 during the weekends, although it drops below 4% during weekdays. But that still leaves 90+ % of all internet users on just IPv4. Within a few years that number will very likely be much lower, but there's still going to be significant numbers of people stuck on IPv4 while network operators have to start using more and more problematic mechanisms to keep IPv4 running without access to new address space.

6to4 was all about getting the first IPv6 users connected. Within some years, we'll be looking at getting the last IPv4-only users connected to IPv6, which is a different challenge.

> I would rather people had good IPv4 connectivity and no IPv6 connectivity, than them having good IPv4 connectivity, so-so IPv6 connectivity, people learning that and turning off IPv6 to fix their problem, and then when their operator rolls out proper IPv6, the end user system still has IPv6 turned off.

But it's not a choice between those two. Obviously many people will continue to enjoy good IPv4, and at this time and in the immediate future you can get by quite well without IPv6. (Although I can only log into my computer at home over IPv6 currently.)

However, IPv4 is going to get worse as CGN gets rolled out and especially as the number of users behind an IPv4 address increases. At some point, lack of port numbers is going to become problematic for some users.

There's no reason tunnels have to be so-so. A tunnel discovery mechanism would be helpful in finding the best tunneling option that a user can use at a given time using a given connection to the network, and of course if no good options are available, the system can say so and no IPv6 tunnel is set up. By having the smarts inside a limited number of servers rather than in millions of individual home gateways, it's easy to react to changing circumstances.

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Thu, 13 Nov 2014 19:50:42 +0000

> automatic IPv6 tunneling thingie that might work". 6to4 was nice to get a way
> for people to try out IPv6. There are now tens of millions of households with
> native IPv6 connectivity in the world without much ill effects. It's now up to
> the operators to roll out to the rest of the mass market with proper reliability
> and quality, and this is happening.
> 
> learning that and turning off IPv6 to fix their problem, and then when their
> operator rolls out proper IPv6, the end user system still has IPv6 turned off. 

+1
We're only now getting to a point where many (but nowhere near all) CE routers being put on retail shelves today properly support requirements for native IPv6 (RFC 7084, and IPv6 CE Router certification from IPv6 Forum). I'd really, really love for the CE router vendors to focus on getting that number up to "all CE routers on retail shelves". Anything else is an unnecessary and likely-to-delay-things distraction at this point, IMO. It would be yet another indicator that IPv6 requirements aren't finished and the vendors should continue to wait until the requirements really, really aren't a moving target.
Barbara

---

From: Gert Doering <[email-masked]>
Date: Thu, 13 Nov 2014 20:56:36 +0100

Hi,

On Thu, Nov 13, 2014 at 09:50:31AM -1000, Iljitsch van Beijnum wrote:
> There's no reason tunnels have to be so-so. 

I'm wondering who would provide these tunnels in the future, and what would
their business model be.

This is a real question.  Today, you can get tunnels from Sixxs nodes
(enthusiasts, which might eventually consider the IPv6 deployment issue
to be "solved" and stop running tunnel nodes), HE.NET (it's good marketing,
but eventually the costs involved might win over the marketing gain?), 
and others that run tunnels for various reasons that might not be there
in 2-3 years...

6rd is a very specific tunnel technology that has answers to that question
("I want to roll out IPv6 to my customers but part of my infrastructure
cannot do it.  So I provision my CPEs to do 6rd, and take responsibility
to run the 6rd relay") - but as with 6to4 relays, what is the incentive to 
run a tunnel broker in 2-3 years from now?  You won't get paid (users do
not pay for IPv6, otherwise they would go and choose ISPs that provide
v6)...

Gert Doering
        -- NetMaster

---

From: George Michaelson <[email-masked]>
Date: Thu, 13 Nov 2014 09:59:39 -1000

HE.net tunnels cause severe distortion of RTT. They are anything but local
termination for much of the world, and whilst I applaud their role in
promoting V6 adoption, its a poor fit to the money flow of IPv4
relationships for many people. I should know: I have one in Australia, and
my nearest terminations are SG/HK (which is via the US and trombones) or
the US (which is 200+ms delay from me)

I cannot speak to SIXX but I suspect it too distorts the packetflow and
causes a disparate RTT compared to native.

6rd is homed in your own ISP. its almost 1:1 congruent with your normal RTT
for native IPv6 endpoints.

When we say 'tunnels are bad' its not because GRE randomly corrupts bits:
its because the effects on the end-to-end model are bad. The badness
varies, but the consequence of a tunnel is usually not what you wanted in
reality, unless the sole consideration was connectivity per se, not its
quality, compared to V4

-G

---

From: "Carlos M. Martinez" <[email-masked]>
Date: Thu, 13 Nov 2014 10:03:46 -1000

I agree, the problem is not with the tunnels per-se, but rather with the
traffic flows they help create.

6rd is a good stepping stone, homed locally performs at almost native
speeds and the added delay is negligible (as perceived from a
residential DSL service)

regards

Carlos

On 11/13/14 9:59 AM, George Michaelson wrote:
> HE.net tunnels cause severe distortion of RTT. They are anything but
> local termination for much of the world, and whilst I applaud their role
> in promoting V6 adoption, its a poor fit to the money flow of IPv4
> relationships for many people. I should know: I have one in Australia,
> and my nearest terminations are SG/HK (which is via the US and
> trombones) or the US (which is 200+ms delay from me)
> 
> 6rd is homed in your own ISP. its almost 1:1 congruent with your normal
> RTT for native IPv6 endpoints.
> 
> When we say 'tunnels are bad' its not because GRE randomly corrupts
> bits: its because the effects on the end-to-end model are bad. The
> badness varies, but the consequence of a tunnel is usually not what you
> wanted in reality, unless the sole consideration was connectivity per
> se, not its quality, compared to V4
> 
> -G
> 
> On Thu, Nov 13, 2014 at 9:56 AM, Gert Doering <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
>     Hi,
> 
>     to be "solved" and stop running tunnel nodes), HE.NET
>     <http://HE.NET> (it's good marketing,
>     Tel: +49 (0)89/32356-444 <tel:%2B49%20%280%2989%2F32356-444>       
>        USt-IdNr.: DE813185279
> 
>     [email-masked] <mailto:[email-masked]>

---

From: Jeroen Massar <[email-masked]>
Date: Thu, 13 Nov 2014 21:55:34 +0100

> causes a disparate RTT compared to native.

Hence why having a large variety of PoPs in disjunct networks is
important and then terminating the tunnel as close as possible to the
point of origin keeping network paths short and latency low.

Note that OCCAID provides 2 PoPs in Australia through SixXS:
 https://www.sixxs.net/pops/occaid/

And AARNet still apparently runs a GogoServer based broker there too.

I wonder what you exactly mean with "severe distortion in RTT" (hence
the subject change) any more details?

> RTT for native IPv6 endpoints.

6rd are just proto-41 tunnels that are locally terminated.

XS4all.nl still runs a tunnel broker that is based on the IPng.nl
scripts that Pim van Pelt wrote back then (and likely majorily
overhauled and likely ported to some Juniper or so by now.

Their tunnels where thus also 'local' to the ISP and had the same
properties.

Unfortunately not every ISP has the expertise or the willingness to set
up a system like that. We made the SixXS model therefor "white label" on
request by a lot of folks in the RIPE community back then. That way,
they just connected the box and got a local PoP that matches these
properties.

> se, not its quality, compared to V4

Over the 15+ years that I have been using IPv6 tunnels I rarely had big
issues with them, but this because there are a lot of well connected
PoPs in the area. Latency increase over IPv4 is minimal most of the time
too.

The only major problem that pops up once in a while is broken PMTUD at
some site or another. Typically a misconfig or a broken device (as
currently is happening). Work on that is underway...

Greets,
 Jeroen

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 13 Nov 2014 22:03:51 +0100 (CET)

> There's no reason tunnels have to be so-so.

Sure, and there is no reason 6to4 wouldn't work either. Now, history has 
proven that it doesn't work very well in reality and I'd rather learn from 
history instead of trying again something we know didn't work very well 
and there is little reason to believe that something new we come up with 
along similar lines would work a lot better.

I have a HE tunnel home until my ISP gets me native IPv6. I tried managed 
IPv6 tunnels in the 2000:s until we could get native IPv6. Native IPv6 is 
superior to any of this, and I venture to say that even HE tunnel, being 
run by professionals, work even close to as well as native IPv6.

Tunnels are an increased complexity and they introduce more mechanisms 
that can break, often in hard to diagnose ways. Let's not throw brain 
cycles on this that could be spent better on deploying native IPv6.

---

From: George Michaelson <[email-masked]>
Date: Thu, 13 Nov 2014 11:16:10 -1000

Sure. ping6 algebras.org and ping algebras.org. I just did this from
Dallas, and the effective difference in RTT was 50ms, and about 1/3 of the
RTT extra compared to V4. 33% is a heavy penalty.

I did the same from Hetzner in Germany, the effect was reversed. Why?
because HE peers directly with the transit behind my Hetzner node, so its a
good path embedded in HE. So the initial look from one asset I have is
"wow: v6 is great" but from another asset outside that family.. its 100%
inverse.

Thats what I mean by RTT distortions. The effect of the tunnel is highly
variable RTT depending on where you are, compared to V4.

the 6 is an HE tunnel, whose endpoint is the US west coast. Had I chosen
their SG or HK presentation, I'd be even worse because of the trombone of
the path from AU to SG being via the USA.

I entirely understand your retort will be "so use gogo" or "so use sixx in
AU via AARNET" but thats the thing: the reason I took HE is because when I
searched google for IPv6 tunnel FreeBSD (I can't remember the exact search)
the top hit was HE and it was simple to configure.

shooting yourself in the foot is very easy with hand-install tunnels.

---

From: Jeroen Massar <[email-masked]>
Date: Thu, 13 Nov 2014 22:39:32 +0100

On 2014-11-13 22:16, George Michaelson wrote:
> Sure. ping6 algebras.org <http://algebras.org> and ping algebras.org
> <http://algebras.org>. I just did this from Dallas, and the effective
> difference in RTT was 50ms, and about 1/3 of the RTT extra compared to
> V4. 33% is a heavy penalty.

64 bytes from geeohgeegeeoh-1-pt.tunnel.tserv15.lax1.ipv6.he.net:
icmp_seq=3 ttl=52 time=317 ms

64 bytes from ec2-54-79-110-87.ap-southeast-2.compute.amazonaws.com
(54.79.110.87): icmp_seq=1 ttl=49 time=368 ms

Yeah, I see those 50ms in the wrong direction ;)

> because HE peers directly with the transit behind my Hetzner node, so
> its a good path embedded in HE. So the initial look from one asset I
> have is "wow: v6 is great" but from another asset outside that family..
> its 100% inverse.

Yep. But that is purely a peering policy and or just BGP bad/good luck.

Can even happen with an ISP's IPv4 and IPv6 prefixes. Though because of
likely getting upstream/transit from the same place there is a higher
chance for it being the same.

> variable RTT depending on where you are, compared to V4.

Understood.

> their SG or HK presentation, I'd be even worse because of the trombone
> of the path from AU to SG being via the USA.
> 
> I entirely understand your retort will be "so use gogo" or "so use sixx
> in AU via AARNET" but thats the thing

Note that there is no SixXS PoP in AARnet.
AARnet does have a GogoServer though, which is not Freenet6.

But for the SixXS nodes in Australia (provided by OCCAID):

aubne01:~$ ping 54.79.110.87
PING 54.79.110.87 (54.79.110.87) 56(84) bytes of data.
64 bytes from 54.79.110.87: icmp_req=1 ttl=56 time=17.8 ms

That is quite close to there...

ausyd01:~$ ping 54.79.110.87
PING 54.79.110.87 (54.79.110.87) 56(84) bytes of data.
64 bytes from 54.79.110.87: icmp_req=1 ttl=56 time=2.89 ms

That is likely in the same datacenter.... ;)

Though that is just the first hop, the rest of the circuit might be
completely different from what you have currently. YMMV etc.

> the reason I took HE is because
> when I searched google for IPv6 tunnel FreeBSD (I can't remember the
> exact search) the top hit was HE and it was simple to configure.

As HE provides 'normal proto-41 tunnels' any provider doing proto-41
config is the same btw. (See the list on wikipedia for a full overview)

That said: use the best endpoint and get great connectivity.

Indeed, I won't say "use SixXS, it is awesome" (even though it is) but I
do say to use the best connectivity one can get, if that is native
great, if it is something other than SixXS also excellent.
(And as Gert mentioned, one day those PoPs will disappear too, or I just
get tired of approving tunnels and fixing things all the time)

Since having bashed Hurricane a lot years ago when they had a lot of
broken and bad connectivity with very bad peering setups they have
improved a lot (clearly the bashing helped :) thus likely the
connectivity is just fine.

> shooting yourself in the foot is very easy with hand-install tunnels. 

That rarely happens if you have been configuring thousands of them ;)

Greets,
 Jeroen

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 13 Nov 2014 21:46:24 +0000

> From: v6ops [mailto:[email-masked]] On Behalf Of Alexandru Petrescu
> Sent: Wednesday, November 12, 2014 9:10 PM
> To: Iljitsch van Beijnum; [email-masked] WG
> Subject: Re: [v6ops] Bar BoF on a 6to4 replacement?
> 

Others have correctly commented that DHCPv6 PD is widely used by
service providers over access links to provide their customers' service.

> should be no problem in using it.

DHCPv6 PD over a tunnel is exactly the case for AERO, yes.

Thanks - Fred

---

From: Jeroen Massar <[email-masked]>
Date: Thu, 13 Nov 2014 22:48:24 +0100

> I'm wondering who would provide these tunnels in the future, and what would

These services already exists, they are called "VPN Providers".

Typically used for the more shady reasons, but they can in more
professional forms be used for what most people tend to use IPv6 for:
getting connectivity with a public IP address.

> to be "solved" and stop running tunnel nodes)

That is the end goal indeed :)

[..]
> but as with 6to4 relays, what is the incentive to
> run a tunnel broker in 2-3 years from now?

Getting IPv6 out there, learning from the problems, letting people use it.

But as when closing down the 6bone: we are today already long past the
point where these experiences are needed.

We know what is broken and we know how to approach it.

Tunnel Brokers, 6rd etc are transition tech. Hopefully everybody will
have native at one point or another.

> You won't get paid (users do

VPN providers are proof that people pay for tunneled connectivity.

Heck, most of those are stuffing users behind NAT and not even giving
public IP addresses away to "protect the anonymity of the user"...

And there are tunnel brokers doing it for fun and free ;)
It is all magic :)

Greets,
 Jeroen

---

From: Jeroen Massar <[email-masked]>
Date: Thu, 13 Nov 2014 22:56:41 +0100

On 2014-11-13 20:50, Iljitsch van Beijnum wrote:
[..big snip..]
> However, IPv4 is going to get worse as CGN gets rolled out and
> especially as the number of users behind an IPv4 address increases.
> At some point, lack of port numbers is going to become problematic
> for some users.

Tunnels will not fix this problem.

Actually: there are German providers doing DSLite/AFTR kind of setups.

Their IPv6 at one point was so broken that various folks started doing
AYIYA tunnels over the CGN to get working IPv6...

Another effect that was seen that apparently some of the CGN boxes would
give a public IP to a user when doing non-TCP/UDP "to maintain
compatibility with VPNs", and thus people would use a proto-41 tunnel
and just keep it pinging so that they got a public IPv4 address ;)

> There's no reason tunnels have to be so-so. A tunnel discovery
> mechanism would be helpful in finding the best tunneling option that
> a user can use at a given time using a given connection to the
> network, and of course if no good options are available, the system
> can say so and no IPv6 tunnel is set up. By having the smarts inside
> a limited number of servers rather than in millions of individual
> home gateways, it's easy to react to changing circumstances. 

Such a system need to be cooperative. Not going to happen.

Also note that latencies and throughputs change. Thus the moment you
would do a measurement the properties would be X and moments later Y.

Are you going to move the user or just keep them on the same box?

Too many issues that even I don't want to bother solving...

As a anecdote: SixXS used to ping all the tunnel endpoints for selecting
the 'best' PoP. Hence a user providing an IPv4 address when requesting a
tunnel would suddenly get a stream of pings from 50+ sources (and
firewall alarms suddenly went off ;).

The result though is that the user would only be eligible to get 1 PoP
as there was only 1 PoP serving their area. Or that PoP had the best
ping but actually was shoveling a lot of traffic already, thus it was
better to pick another one etc. Hence why we just manually approve the
tunnels and allow the users to prefer a PoP that they are eligle for.

This approval process is a big scaling issue but it does also allow
people from making mistakes, eg "I have a Fritz!Box" and then requesting
an AYIYA tunnel...

(The TIC server fixes up that mistake when it sees a Fritz!Box client
requesting a tunnel though ;)

Greets,
 Jeroen

---

From: Alexandru Petrescu <[email-masked]>
Date: Thu, 13 Nov 2014 23:02:17 +0100

Hi Iljitsch, just some clarification.

Le 13/11/2014 20:21, Iljitsch van Beijnum a écrit :
>> encrypted or authenticated exchange, with someone who is on the
>> other side of the globe, someone non-accountable in any particular
>> way.  A matter of trust.
>
> Yes, it would be somewhat risky to put the cloud credentials that
> give access to your entire digital life inside a $50 home gateway. So
> it would be good to require OAuth (RFC 6749). On the other hand, that
> could be quite a barrier for implementers.
>
> However, I'd say that HTTP digest authentication is secure enough to
> protect an account that is only used to set up a tunnel.

Sorry, I didnt mean to challenge the strength of such and such security 
protocol.  I meant to say that a new mechanism involving two endpoints 
(e.g. DSL box, and tunnel endpoint on the other side) should offer 
enough confidence to the DSL box about the legitimacy and goodwill of 
the tunnel endpoint.  Something like what the 3rd party CAs offer, but 
not that.

>> I guess it is problematic.  I am not aware of any operator of DSL
>> or cellular links that offers DHCPv6 Prefix Delegation service (a
>> Server that replies positively to requests for prefixes).
>
> It doesn't matter what the network operators do. If they provide
> native IPv6, then the whole effort is moot. What counts is what we
> can reasonably ask home gateway vendors to implement.

Then I think it is reasonable to think that a home gateway includes a 
DHCPv6 Client able to issue Prefix Delegation requests and treat replies 
accordingly.

Alex

---

From: Alexandru Petrescu <[email-masked]>
Date: Thu, 13 Nov 2014 23:22:55 +0100

> have native at one point or another.

Goal - any new replacement of 6to4 must be future proof: avoid 
renumbering when 6to4-replacement dies out.  (this is what current 
tunnel brokers dont offer, they force one into a lockin going away from 
tunnel broker to native IPv6 one has renumber).

Alex

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 13 Nov 2014 12:31:54 -1000

On 13 Nov 2014, at 12:22, Alexandru Petrescu <[email-masked]> wrote:

> Goal - any new replacement of 6to4 must be future proof: avoid renumbering when 6to4-replacement dies out.  (this is what current tunnel brokers dont offer, they force one into a lockin going away from tunnel broker to native IPv6 one has renumber).

Are you saying that you want to be able to move from a tunnel to native without renumbering?

I don't think that's even a minor goal; renumbering is not a big deal in a home environment.

---

From: Alexandru Petrescu <[email-masked]>
Date: Thu, 13 Nov 2014 23:41:19 +0100

Le 13/11/2014 23:31, Iljitsch van Beijnum a écrit :
>> tunnel brokers dont offer, they force one into a lockin going away
>> from tunnel broker to native IPv6 one has renumber).
>

Yes.  My situation currently is that my network is on 6to4, it seems to 
get discarded, and the trustful ISP is not _yet_ offering native.

> I don't think that's even a minor goal; renumbering is not a big deal

Depends on the size of the home environment.  Considering the operators 
currently owning 2 devices but announcing to put and control much more 
devices in your home for IoT apps, one thinks avoiding renumbering could 
only be beneficial.

Alex

---

From: Jeroen Massar <[email-masked]>
Date: Thu, 13 Nov 2014 23:52:16 +0100

On 2014-11-13 23:22, Alexandru Petrescu wrote:
[..]
> tunnel broker to native IPv6 one has renumber).

Please solve this problem for native IPv6, provided by an ISP, first.

Then we can apply the same for Tunnel Brokers...

Just in case you wonder: for the time being it is called PI.

Most Tunnel Brokers give out static prefixes btw; I have had some
already for well over 10 years...

On 2014-11-13 23:41, Alexandru Petrescu wrote:
> Yes.  My situation currently is that my network is on 6to4,
> it seems to get discarded,

What gets "discarded"?

> and the trustful ISP is not _yet_ offering native.

What do you mean with "the trustful ISP"?

> Depends on the size of the home environment.  Considering the
> operators currently owning 2 devices but announcing to put and
> control much more devices in your home for IoT apps, one thinks
> avoiding renumbering could only be beneficial.

I have a rather extensive home network. Renumbering though means I
change the prefix in my config file, and then let my scripts spit out
new DNS, new firewall settings etc and push that to the relevant boxes.

Renumbering is easy for the portions that are in your own control (yeah,
for total control of all that is involved).

Renumbering is extremely hard when you need to tell other entities what
the new settings are.

Oh, and no, I don't want to renumber my network, I love my static prefix.

Greets,
 Jeroen

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 13 Nov 2014 22:52:58 +0000

Hi,

> tunnel broker to native IPv6 one has renumber).

To amplify this point, tunnels are for more than just IPv6 transition. They are
useful also for:

  - mobility management
  - routing control
  - security (e.g., VPN)
  - multihoming
  - traffic engineering
  - renumbering avoidance
  - etc.

You get all of this (plus v6/v4 transition, of course) with AERO. Indeed,
AERO can be a one-stop-shopping solution for any tunnel application.

Thanks - Fred

---

From: Jeroen Massar <[email-masked]>
Date: Fri, 14 Nov 2014 00:00:43 +0100

> AERO can be a one-stop-shopping solution for any tunnel application.

AYIYA handles those fine too. Tested and in use by over 20k people,
available from quite a number of CPEs too ;)

OpenVPN works perfectly fine for probably millions of people.

Don't call those things "tunnels" though, just name them by their
commercial name: VPNs.

Greets,
 Jeroen

---

From: Alexandru Petrescu <[email-masked]>
Date: Fri, 14 Nov 2014 00:07:49 +0100

Jeroen - you make up a few pretty good questions, I am happy.  If I dont 
keep up sorry.

>> it seems to get discarded,

deprecated 6to4 seems to become by meeting discussion.

> What do you mean with "the trustful ISP"?

An ISP that has same upper organization that my organization.  In 
private you pointed me to it, you know what I mean.

> new DNS, new firewall settings etc and push that to the relevant boxes.

Well by renumbering I mean the Renumbering RFC which uses a form of 
Router Advertisement propagation.

You have scripts - great, I am happy for you.

> Oh, and no, I don't want to renumber my network, I love my static prefix.

:-)

Alex

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 13 Nov 2014 23:31:02 +0000

Hi Jeroen,

> Sent: Thursday, November 13, 2014 3:01 PM
> To: Templin, Fred L
> Cc: [email-masked] WG

I forgot to mention also:

  - route optimization
  - distributed mobility management

With AERO, there can be many 100's of Servers, and the Clients get the same
service no matter which Server(s) they associate with. The Server-to-Client
associations are communicated to Relays which connect the AERO link to the
rest of the IPv6 Internet. Server association is through DHCPv6 PD messaging,
and Route optimization is through IPv6 ND messaging the same as for any link.

> OpenVPN works perfectly fine for probably millions of people.

OpenVPN is indeed the target integration platform for the AERO Client.
Only, it is used whether or not the tunnel also applies IPsec - it would
also be used for simple IPv4/UDP/IPv6 encapsulation.

> Don't call those things "tunnels" though, just name them by their

Tunnels are VPNs and VPNs are tunnels, sure, but I don't see a reason for
making such a distinction for the purpose of this discussion.

Thanks - Fred

---

From: Jeroen Massar <[email-masked]>
Date: Fri, 14 Nov 2014 01:43:20 +0100

[..]
> deprecated 6to4 seems to become by meeting discussion.

But there are MANY alternatives which will work much better.

Thus what is the problem?

> private you pointed me to it, you know what I mean.

Kick your internal organization. It is 2014. Not 1996.

Also the upstream organization (RENATER) has been doing native IPv6 for
many many many years thus they will be more than happy to deliver it.

The IETF cannot fix politics inside organizations. You will have to
fight for that.

[..]
> Router Advertisement propagation.

I have never ever seen anybody use that.

> You have scripts - great, I am happy for you.

Any decent system administrator or network operator who is lazy as they
should be will do that.

Systems like rancid, puppet etc exist for many many reasons.

(I just rsync stuff with a /etc/init.d/... after it ;)

Greets,
 Jeroen

---

From: Jeroen Massar <[email-masked]>
Date: Fri, 14 Nov 2014 01:50:40 +0100

> I forgot to mention also:

Does it get a BGP feed to make these decisions?

>   - distributed mobility management

The 'distributed' part comes from flooding these details to network.

How does that scale with say 100.000 clients?

> With AERO, there can be many 100's of Servers,

And who would operate and coordinate these?
How are these connected to the Internet?
What kind of address would they use and where do they get these from?

And how would a client be configured, needs anycast?

> and the Clients get the same
> service no matter which Server(s) they associate with.

Peer-To-Peer networks do the same trick. Firewalls tend to not like them.

> The Server-to-Client
> rest of the IPv6 Internet.

How is this connection done? Are you peering at every major IX in the
world? Or are you announcing a Anycast prefix everywhere?

> Server association is through DHCPv6 PD messaging,

What is the decision process for "optimizing a route"? How is this
better than the decisions made by actual live operators and BGP?

> OpenVPN is indeed the target integration platform for the AERO Client.

OpenVPN is not a platform, it is a protocol implemented by a client+server.

> Only, it is used whether or not the tunnel also applies IPsec

OpenVPN does not do IPSec on the outside, it uses its own crypto.
The tunneled packets might be IPsec though.

> - it would

How?

> making such a distinction for the purpose of this discussion.

As the subject was "6to4 replacement" we are talking about IPv6 tunnels.

VPNs are a specific case of provider-provided networking.

Typically with money involved and typically requiring credentials.

Greets,
 Jeroen

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 14 Nov 2014 02:09:50 +0000

Hi Jeroen,

> Sent: Thursday, November 13, 2014 4:51 PM
> To: Templin, Fred L
> Cc: [email-masked] WG
> Subject: AERO again (Was: Bar BoF on a 6to4 replacement?)
> 

No, route optimization is coordinated between Clients, Servers and Relays and does
not interact with the global Internet BGP routing system. However, Servers and
Relays do engage in a BGP routing protocol instance that is kept separate and is
used only to track the association of Client prefixes with Servers. So, for example.
if a Client has an AERO Client Prefix (ACP) P and associates with Server S, the AERO
BGP routing system includes the routing entry P->S.

In terms of connecting to the larger IPv6 Internet, the AERO link has an aggregated
(short) AERO Service Prefix (ASP) A that it announces into the global BGP routing
system. This prefix never changes and does not become de-aggregated due to
mobility, etc.

> The 'distributed' part comes from flooding these details to network.

I didn't understand that, but there is no flooding. Clients simply discover
Servers that are close by and select one or more Servers to provide them
with the service.

> How does that scale with say 100.000 clients?

Each AERO link can scale to as many ACPs as can be represented in the internal
BGP routing protocol instance - let's say that it can handle up to 1M ACPs.  But,
there may be many such AERO links.

> And who would operate and coordinate these?

The AERO link administrative authority.

> How are these connected to the Internet?

As ordinary IPv4 hosts in the public Internet.

> What kind of address would they use and where do they get these from?

In the case of a 6to4 replacement scenario, they would use the public IPv4
addresses they get from their ISPs. But, the AERO link would not necessarily
be bound to just one ISP; it can include Servers that are distributed across
many ISPs.

> And how would a client be configured, needs anycast?

Anycast is certainly possible, but the expected mechanism is through DNS
resolution of a FQDN like "linkupnetworks.example.com". So, the Client
needs to be provisioned with the FQDN.

> Peer-To-Peer networks do the same trick. Firewalls tend to not like them.

Didn't get that.

> world? Or are you announcing a Anycast prefix everywhere?

The AERO link would be represented as an AS to the global BGP routing system.
Either the AERO Relays themselves or other BGP routers advertise the ASPs to
their global BGP routing system peers.

> better than the decisions made by actual live operators and BGP?

If a Client A discovers that Client B is attached to the same AERO link, Client A
initiates route optimization, and the RO message is conveyed through Servers
(and possibly also a Relay) as a chain-of-trust whereby Client B can know that
it can accept messages directly from A. It is asymmetric, so a corresponding
RO would also have to happen in the reverse direction. Both ways, the AERO
link Servers and Relays provide a chain-of-trus

> OpenVPN is not a platform, it is a protocol implemented by a client+server.

OpenVPN is also a software application that runs on a Client. What I
meant by "platform" is that the AERO functions would be added to
the OpenVPN code base and then run on a Client device.

> The tunneled packets might be IPsec though.

I don't really care what the exact securing mechanisms are. What I care
about is the AERO control message signaling regardless of the specific
encapsulation.

> > also be used for simple IPv4/UDP/IPv6 encapsulation.

By not including security encapsulations when they are not needed.

> As the subject was "6to4 replacement" we are talking about IPv6 tunnels.

Call it tunnels then, I don't care. AERO tunneling is based on IP/UDP/IP when
it needs to go through filtering middleboxes or on IP/IP when it does not.
If security is needed, it would then go as IP/IPsec/IP, but that is only for
environments that need IP security.

> Typically with money involved and typically requiring credentials.

AERO is not constrained to IPsec-like VPNs only - for the public Internet case,
we would just use IP/IP or IP/UDP/IP encaps.

Thanks - Fred

---

From: Alexandru Petrescu <[email-masked]>
Date: Fri, 14 Nov 2014 03:15:05 +0100

> Thus what is the problem?

The ones you suggest involve renumbering later - painful.

> Kick your internal organization. It is 2014. Not 1996.

Arguing at an organization takes more time than the flash depreciation 
at IETF...

Sure.

> The IETF cannot fix politics inside organizations. You will have to

This is not politics.  This is simple disconnect between what people 
think at IETF it is good for end users.

> I have never ever seen anybody use that.

I guess the authors of the RFC at least have used it.

> Systems like rancid, puppet etc exist for many many reasons.

Huh?  Does it compile for a 1cmx1xm LTE embedded device, for example?

> (I just rsync stuff with a /etc/init.d/... after it ;)

Good to know thanks.

Alex

---

From: Jeroen Massar <[email-masked]>
Date: Fri, 14 Nov 2014 04:00:04 +0100

> The ones you suggest involve renumbering later - painful.

As you chose to use 6to4 you will have to renumber anyway one day. 6to4
is a transition technique, hence bound to go away.

Note that technically you do not have to renumber, you can keep on using
that 6to4 prefix, just not with a default pointing to the Internet
anymore, especially not that anycast address.

Also note that when changing ISP one will have to renumber, unless you
are your own ISP and play the PI game you will not be getting away from
that.

Hence why becoming an LIR and getting your own ASN, own IPv6 PI and a
last chunk of IPv4 is very popular, even home users do that already.

Another note there is that most CGN/DSLite providers have are delegating
prefixes that rotate every 24 hours, for "accounting" reasons or "your
privacy and anonimity" but in reality so that folks won't run servers in
the cheap space and need to buy hosting instead.
> Arguing at an organization takes more time than the flash depreciation

This draft will become an RFC... vendors won't do much, existing devices
will keep on doing what they do as they do not get updated.

Maybe operators will start stopping their relays, but that is it. That
will take a couple more years though for all that to fade away.

> This is not politics.

In the end everything comes down to politics... if you like it or not.

> This is simple disconnect between what people

I think "the IETF" is of the opinion that native IPv6 is the way to go.

This is also the message being sent out with deprecating 6to4 which is
just one of many transition mechanisms.

As you state "end user" btw, you should be contacting your network team
to get their act together. They kind of had already 15+ years of warning
that this was coming and the hype machine has been doing proper
information distribution and pushing folks to do this IPv6 thing.

> I guess the authors of the RFC at least have used it.

I'll refer to a previous message in this thread to answer your question:
 http://www.ietf.org/mail-archive/web/v6ops/current/msg20524.html

Demonstrating that the folks who write the drafts do not always use
their own tools...

> Huh?  Does it compile for a 1cmx1xm LTE embedded device, for example?

Contact the vendor of your device for details on how to properly manage
them. Likely though it is just DHCP that you'll use for it or some
custom Enterprise tool that can set all kinds of settings.

Greets,
 Jeroen

---

From: Alexandru Petrescu <[email-masked]>
Date: Fri, 14 Nov 2014 05:59:12 +0100

> is a transition technique, hence bound to go away.

Yes, I prefer to reunmber only once.

If one depreciates 6to4 now, and as I can not get native now, I am found 
in a situation to renumber twice - once for the tunnel brokers and once 
when I go native.  Right?

> anymore, especially not that anycast address.

What does it mean?  Somebody else than that anycast address could 
forward my 6to4 traffic, for free?  Somebody nearby?

> are your own ISP and play the PI game you will not be getting away from

Changing ISP is not an option.

> will take a couple more years though for all that to fade away.

The past experience of deprecating site-locals was much faster...

> I think "the IETF" is of the opinion that native IPv6 is the way to go.

Well that is a good opinion.

But bringing to life is not easy, for example because of the established 
fear of the huge number of existing applications IPv4.  People are busy 
maintaining that...

Alex

> Demonstrating that the folks who write the drafts do not always use

Well, on another hand, its easier to write RFCs that remove software 
than writing RFCs that need new software...

> custom Enterprise tool that can set all kinds of settings.

YEah I did - IPv6 is not important, the applications are important.  How 
many new IoT applications can we imagine for them such that more get 
sold.  Most work without IPv6, IPv4 is sufficient.

Alex

---

From: Jeroen Massar <[email-masked]>
Date: Fri, 14 Nov 2014 09:28:54 +0100

> in a situation to renumber twice - once for the tunnel brokers and once

Quite likely.

You could have spent the last several years and the coming couple of
years before 6to4 is really gone fighting to get native IPv6.

Remember: it is 2014. It has kind of been announced for well over a
decade already.

Also, you are in an environment where your "ISP" (RENATER.fr) provides
native IPv6 for a long time already.

Or you can do what some people do: NAT IPv6 when it leaves the perimeter ;)

> forward my 6to4 traffic, for free?  Somebody nearby?

No. Nobody will forward you traffic.

You just will have local connectivity in your local network.

> Changing ISP is not an option.

Going from 6to4 to something else is effectively changing ISP. Currently
your ISP is a random one providing "6to4".

Swap to the one that does "native".

[..]
> The past experience of deprecating site-locals was much faster...

Site-locals where local and really nothing used it.

[..]
> fear of the huge number of existing applications IPv4.  People are busy

You mean "maintaining the fear"?

Applications will be moving over, with the IPv4 pool as good as gone
they have to.

Greets,
 Jeroen

---

From: Keith Moore <[email-masked]>
Date: Wed, 19 Nov 2014 13:26:49 -0500

If there was actually a bar BOF on a 6to4 replacement, would someone 
care to summarize?

(private mail is fine if you don't think it belongs on the list)

thanks!

Keith