Subject: multi-homing for provider-assigned IPv6 addresses

From: Chris Bowers <[email-masked]>
Date: Tue, 15 Mar 2016 13:14:09 +0000

RTGWG,

We scheduled a significant amount of time in Buenos Aires for a discussion of multi-homing for provider-assigned IPv6 addresses for enterprise networks as well as homenet.  I want to explain the motivation for this and provide some background on the topic.  And hopefully spark some discussion on the list before Buenos Aires, as well.

RTGWG adopted draft-ietf-rtgwg-dst-src-routing last October in the context of supporting multi-homing for provider-assigned IPv6 addresses in homenet.  In Yokohama, the v6ops WG had a lengthy discussion about the need for a solution to support multi-homing for provider-assigned IPv6 addresses for enterprise networks in general (as opposed to just homenet).  This discussion took place in the context of draft-ietf-v6ops-design-choices.  That discussion can be found at:

https://youtu.be/VzH7yqqGiGc?t=5835

This discussion led to the email (copied below) from Fred Baker in his role as v6ops co-chair to the chairs of several working groups in which drafts related to this topic are being discussed.  

Our meeting in Buenos Aires includes a 20 minute time slot to discuss the background and motivation of this request from v6ops.  The topic of multi-homing for provider-assigned IPv6 addresses has a long history with many documents written, so I thought it would be useful to highlight a few of the more recent documents that I found to be particularly useful reading.

RFC 7157 "IPv6 Multihoming without Network Address Translation"
https://datatracker.ietf.org/doc/rfc7157/

draft-ietf-6man-multi-homed-host-06 "Routing packets from hosts in a multi-prefix network"
https://datatracker.ietf.org/doc/draft-ietf-6man-multi-homed-host/

In addition, it would obviously be useful read or re-read the RTGWG document on this topic:
https://datatracker.ietf.org/doc/draft-ietf-rtgwg-dst-src-routing/

There following expired document is also helpful.
https://www.ietf.org/archive/id/draft-baker-rtgwg-src-dst-routing-use-cases-01.txt

And finally, this document is useful to understand a concrete proposal for how src/dst routing information could be carried in a link-state routing protocol. 
https://datatracker.ietf.org/doc/draft-baker-ipv6-isis-dst-src-routing/

I look forward to a fruitful discussion on this topic on the list and in Buenos Aires.

Thanks,
Chris

-----Original Message-----
From: Fred Baker (fred) [mailto:[email-masked]] 
Sent: Sunday, November 01, 2015 11:03 PM
To: [email-masked]; [email-masked]; [email-masked]; [email-masked]; 6man Chairs <[email-masked]>; [email-masked]
Cc: [email-masked] WG <[email-masked]>; The IESG <[email-masked]>
Subject: PA Address Multihoming in IPv6

This email is being sent in accordance with the v6ops charter, which calls for the working group to communicate operational issues and requirements to working groups that are chartered to address them.

The IETF's current primary recommendation for multihoming of midrange enterprise networks - those that cannot justify the costs and overheads of a PI address and in fact multihome - is to obtain a provider-allocated prefix from each of their upstream networks, and deploy a /64 out of each on each LAN in their networks.

https://tools.ietf.org/html/rfc4213
4213 Basic Transition Mechanisms for IPv6 Hosts and Routers. E. Nordmark,
     R. Gilligan. October 2005. (Format: TXT=58575 bytes) (Obsoletes
     RFC2893) (Status: PROPOSED STANDARD) (DOI: 10.17487/RFC4213)

This has a number of issues, not the least of which is in the back end OSS software, which needs to now scale to a much larger number of prefixes, handle multiple addresses in DNS for servers and perhaps clients, resolve reverse DNS queries, and so on. It also is obviously carrying that much more information in routing.

One outcome of v6ops' discussions this morning was that PI multihoming demonstrably works, but PA multihoming when the upstreams implement BCP 38 filtering requires the deployment of some form of egress routing - source/destination routing in which the traffic using a stated PA source prefix and directed to a remote destination is routed to the provider that allocated the prefix. The IETF currently has no such recommendation, or consensus that it should have. However, enterprise networks are known to delaying operational deployment of IPv6 in part due to the complexities visited upon them and the cost of the back end software upgrades, and this is part of that issue.

Without trying to limit the options available to the working groups in question, I'll point out that options currently on the table include the following. There are also current open source implementations of source/destination and source-specific routing in IS-IS, OSPFv3, and Babel.

https://datatracker.ietf.org/doc/draft-baker-ipv6-isis-dst-src-routing
  "IPv6 Source/Destination Routing using IS-IS", Fred Baker, David
  Lamparter, 2015-10-19

https://datatracker.ietf.org/doc/draft-boutier-babel-source-specific
  "Source-Specific Routing in Babel", Matthieu Boutier, Juliusz
  Chroboczek, 2015-05-27

https://datatracker.ietf.org/doc/draft-ietf-6man-multi-homed-host
  "Routing packets from hosts in a multi-prefix network", Fred Baker,
  Brian Carpenter, 2015-10-15

https://datatracker.ietf.org/doc/draft-ietf-ospf-ospfv3-lsa-extend
  "OSPFv3 LSA Extendibility", Acee Lindem, Sina Mirtorabi, Abhay Roy, Fred
  Baker, 2015-10-08

https://datatracker.ietf.org/doc/draft-ietf-rtgwg-dst-src-routing
  "Destination/Source Routing", David Lamparter, 2015-10-17,

https://datatracker.ietf.org/doc/draft-sarikaya-6man-sadr-overview
  "Source Address Dependent Routing and Source Address Selection for IPv6
  Hosts: Problem Space Overview", Behcet Sarikaya, Mohamed Boucadair,
  2015-08-17

https://datatracker.ietf.org/doc/draft-sarikaya-6man-sadr-ra
  "IPv6 RA Option for Source Address Dependent Routing", Behcet Sarikaya,
  2015-06-08

https://datatracker.ietf.org/doc/draft-sarikaya-dhc-6man-dhcpv6-sadr
  "DHCPv6 Solution for Source Address Dependent Routing", Behcet Sarikaya,
  2015-05-08

https://datatracker.ietf.org/doc/draft-xu-ospf-multi-homing-ipv6
  "Extending OSPFv3 to Support Multi-homing", Mingwei Xu, Shu Yang,
  Jianping Wu, Fred Baker, 2015-10-11,

https://datatracker.ietf.org/doc/draft-baker-rtgwg-src-dst-routing-use-cases

---

From: [email-masked]
Date: Tue, 22 Mar 2016 09:24:41 +0100

Thank you for taking this on!
Marketing alert: If anyone wants to work on implementing source address dependent routing in a high performance software stack, please join the hackathon and register for the "VPP" event.
https://www.ietf.org/registration/MeetingWiki/wiki/95hackathon

Best regards,
Ole

> On 15 Mar 2016, at 14:14, Chris Bowers <[email-masked]> wrote:
> 
> RTGWG,
> 
> _______________________________________________
> rtgwg mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtgwg

---

From: Chris Bowers <[email-masked]>
Date: Thu, 24 Mar 2016 23:34:23 +0000

RTGWG,

It seems to me that most use cases for ipv6 multi-homing with provider-assigned addresses only need to route based on source address when the destination prefix is the default route.  So why not require that source prefixes can only be paired up with the default destination prefix ::/0?  That is, we could require that a router can only route based on the source address after the longest match route lookup based on destination address has reached ::/0.  That way, a router can still route packets local to the site based on destination address.  Only once it is clear that the packet is going to the Internet can the router use the source address to pick the correct CE router to get there. 

I see that there was some discussion about this in 2013 on a thread in homenet and rtgwg in the context of draft-baker-rtgwg-src-dst-routing-use-cases, and a number of different opinions were expressed mainly from the point of view of what is needed for homenet.  At this point, RTGWG also needs to consider the impact of proposed solutions on routing scalability.  On the surface, this simplification would seem to improve the scaling properties of the solution in draft-ietf-rtgwg-dst-src-routing, while covering most use cases.

Regardless of how this discussion turns out, I think the draft itself will benefit from more active discussion of the tradeoffs involved.  

Thanks,
Chris

-----Original Message-----
From: Chris Bowers 
Sent: Tuesday, March 15, 2016 8:14 AM
To: [email-masked]' <[email-masked]>
Subject: multi-homing for provider-assigned IPv6 addresses

RTGWG,

Thanks,
Chris

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Fri, 25 Mar 2016 04:31:50 +0000

> On Mar 24, 2016, at 4:34 PM, Chris Bowers <[email-masked]> wrote:
> 
> It seems to me that most use cases for ipv6 multi-homing with provider-assigned addresses only need to route based on source address when the destination prefix is the default route.  So why not require that source prefixes can only be paired up with the default destination prefix ::/0?

When what one has in mind is an egress route, that probably makes sense. However, it precludes an entire class of use cases mentioned in the use case draft. Why do that?

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Fri, 25 Mar 2016 04:37:01 +0000

> On Mar 24, 2016, at 9:32 PM, Fred Baker (fred) <[email-masked]> wrote:
> 
> 

Let me give you an obvious variant. Imagine, if you will, that I have a PA prefix from each of N ISPs, and therefore a default route to each of N ISPs. Imagine also that I have a particular prefix that I would like to route through via a given one of those N ISPs, in the special case that I happen to have been smart enough to use that source prefix. So now I have N+1 source/destination routes - unless you tie it to default routes.

A premature optimization usually breaks things...

---

From: Anton Smirnov <[email-masked]>
Date: Fri, 25 Mar 2016 12:31:16 +0100

Hi Chris,

 > At this point, RTGWG also
 > needs to consider the impact of proposed solutions on routing
 > scalability.

    For any scalability discussion one has to remember that src-dst 
routing was conceived to IMPROVE routing scalability by avoiding need to 
advertise globally prefixes of small dual-connected networks. We should 
keep in mind that src-dst routing is addressing one of most important 
reasons of why /24 and smaller prefixes are being injected into BGP.

Anton

---

From: Chris Bowers <[email-masked]>
Date: Sun, 27 Mar 2016 16:03:21 +0000

Fred,

I would like to better understand the particular use case offered as a counter-example.  One version of this counter-example is described in Section 2.3 of draft-baker-rtgwg-src-dst-routing-use-cases-01 (Specialized Egress Routing).  I have copied figure 3 from that draft below.

                                                ,-------.
                                             ,-'         `-.
             ,---.            ,-----.      ,'               `.
            /     \    +-+  ,'       `.   /                   \
           /       +---+R+-+   ISP 1   --+---                  \
          /         \  +-+  `.       ,' ;                       :
         ; Customer  :        `-----'   |       The Internet    |
         | Network   |        ,-----.   :                       ;
         :           ; +-+  ,'       `.  \                     /
          \         +--+R+-+   ISP 2   )  \                   /
           \       /   +-+  `.       ,'    `.               ,'
            \     /           `--+--'        '-.         ,-'
             `---'               |              `-------'
                           Some specialized
                              Service

       Figure 3: Egress Routing with a specialized upstream network

If a customer network homed to ISP1 and ISP2 (with PA prefixes S1 and S2) needs to reach a destination prefix (D2) associated with a service offered by ISP2, then the customer network can route all traffic with destination address in D2 to ISP2 using existing destination-based routing.  R for ISP2 would originate a route for D2.  This route for D2 will be used throughout the customer network instead of the default destination route originated by R for ISP1, due the longest prefix match rule.  The counter-example assumes that the host is smart enough to use the source prefix S2 when sending packets to D2, so it will meet the requirement that packets using the special service MUST use the special service's source and destination addresses.

Therefore, it seems to me that pairing a source prefix with an arbitrary destination prefix is not required to support the Specialized Egress Routing use case in draft-baker-rtgwg-src-dst-routing-use-cases-01. In any case, it would be useful to come to agreement about what is required to address the Specialized Egress Routing use case described in draft-baker-rtgwg-src-dst-routing-use-cases-01.

If instead the counter-example use case described below is different from Specialized Egress Routing use case in draft-baker-rtgwg-src-dst-routing-use-cases-01, it would be useful to describe that use case in more detail so that we can evaluate it more thoroughly.

Thanks,

Chris

-----Original Message-----

Sent: Thursday, March 24, 2016 11:37 PM

To: Chris Bowers <[email-masked]>

Cc: [email-masked]

Subject: Re: multi-homing for provider-assigned IPv6 addresses

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Mon, 28 Mar 2016 19:47:13 +0000

On Mar 27, 2016, at 9:03 AM, Chris Bowers <[email-masked]<mailto:[email-masked]>> wrote:

Fred,

                                                ,-------.
                                             ,-'         `-.
             ,---.            ,-----.      ,'               `.
            /     \    +-+  ,'       `.   /                   \

If a customer network homed to ISP1 and ISP2 (with PA prefixes S1 and S2) needs to reach a destination prefix (D2) associated with a service offered by ISP2, then the customer network can route all traffic with destination address in D2 to ISP2 using existing destination-based routing.

Yes, if the packet first goes through a router that is not depicted in the diagram (I didn't describe the interior of the customer network, which could be as simple as a LAN and as complex as you care to imagine). If the only routing is done by the host itself, the packet will go to the router the host considers to be its default router, which is pretty likely to be the other router in this example. That is the subject of draft-ietf-6man-multi-homed-host. Note, BTW, that the picture is essentially NTT's BFLETS configuration, with the exception that (as I understand it) the two ISPs are served by the same managed router, deployed by NTT and configured to route appropriately.

I'm happy to discuss that. I think it misses the point, however. If the use case described isn't a good one, let's think of a better use case, perhaps one that doesn't involve a PA prefix from an ISP. Perhaps the biggest issue with your proposal, which was to take all traffic using a given source prefix to an appropriate egress router, is that it precludes the use of the address to communicate within the customer network. All traffic would dogleg through the egress router, and I'm not sure I see how packets get from that router to a point inside without looping. There has to be a way to prefer to destination-address route some traffic and source-address route some other traffic. The simplest way I can think of is to use both addresses in every route lookup, interpreting a traditional destination route as a source/destination route from ::/0 to the stated destination, or (if there are multiple route tables by source prefix) duplicate the destination route table in each.

I'll take a look at that. Of course, I can't update the draft until next week.

Thanks,
Chris

-----Original Message-----
From: Fred Baker (fred) [mailto:[email-masked]]
Sent: Thursday, March 24, 2016 11:37 PM
To: Chris Bowers <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: multi-homing for provider-assigned IPv6 addresses

> On Mar 24, 2016, at 9:32 PM, Fred Baker (fred) <[email-masked]<mailto:[email-masked]>> wrote:
>
>
>> On Mar 24, 2016, at 4:34 PM, Chris Bowers <[email-masked]<mailto:[email-masked]>> wrote:
>>