Subject: [v6ops] Question on multi-homed nodes and address/route selection

From: Simon Hobson <[email-masked]>
Date: Thu, 2 Mar 2017 09:39:07 +0000

This came up on another list I'm on ...

Take what I believe to be a fairly typical scenario :
A smallish business that's large enough/dependent on connectivity enough to have two internet connections - possibly through different ISPs. But not large enough to justify getting it's own PI address allocation with all the management (and global routing table bloat) that goes with it.

In the IPv4 world it's easy to use RFC1918 private addressing, NAT in the router, and the router makes all the policy decisions as to which (outbound connection) traffic goes out through which connection*. That's fairly easy to manage, and with my work hat on, my employer has several clients with just this setup.

Now introduce IPv6, we have two (or possibly more) separate prefix delegations, each device gets two (or more) addresses. How do people go about setting routing policy in this situation ?
It's no longer the domain of the router, since the source address used by the node determines which connection the traffic needs to be sent via. That would seem to suggest that every node on the network now needs to be given instruction on routing policy. It also means that when the routing requirements change (eg a connection drops), this needs to be communicated to all the nodes.
This seems to be a significant backward step in terms of network management.

* Depending on situation, this may be as simple as "all traffic via X, but if X is down then via Y. Or it may be traffic of type A (eg viop traffic) via Y with all other traffic via X - but if X or Y is down then reroute the traffic that would go through it to the other connection. Or there may be reasons (such as relative cost, speed, and latency/jitter) to send interactive traffic (eg browsing) via one and "bulk" traffic (such as file transfers and mail) via the other.

---

From: "Eric Vyncke (evyncke)" <[email-masked]>
Date: Thu, 2 Mar 2017 10:12:20 +0000

Simon,

How timely :-)

Together with other folks, we are submitting an I-D in the  coming hours about Provisioning Domains (PvD) which addresses your use case of multi-homing in a 'complex case'.

AFAIK, there is another I-D (or even RFC) that recommends hosts to select the default route next-hop based on the source prefix used.

-éric

On 02/03/17 10:39, "v6ops on behalf of Simon Hobson" <[email-masked] on behalf of [email-masked]> wrote:

    

    
    _______________________________________________
    v6ops mailing list
    [email-masked]
    https://www.ietf.org/mailman/listinfo/v6ops

---

From: Simon Hobson <[email-masked]>
Date: Thu, 2 Mar 2017 13:45:26 +0000

> Together with other folks, we are submitting an I-D in the  coming hours about Provisioning Domains (PvD) which addresses your use case of multi-homing in a 'complex case'.

I look forward to looking at it.

> AFAIK, there is another I-D (or even RFC) that recommends hosts to select the default route next-hop based on the source prefix used.

That doesn't really address the issue. It's logical that for whichever address the node picks to use as a source address for an outbound connection, it should use an appropriate outbound route - I would guess, logically a router that advertised the matching prefix as one it routes for.

But you have to take a step back and look at how that source address is picked - because in effect, source address selection for an outbound connection is intrinsically linked to route selection. Ie, if the node/service picks address A then the packet must be routed via connection X, if it picks source address B then the packet must be routed via connection Y.

So it's no good if (just picking an example) I've got a connection that is good for bulk transfers (cheap, maybe slower and/or higher latency/jitter) and want to route my outbound mail via that connection, thus leaving the higher cost but better connection for interactive use (eg browsing) - if the mail service then uses a source address meaning that the packets have to be routed via the latter connection.
I could force the mail service to bind to just one address - but that then means more local config - and it breaks if the latter connection goes down and I need it to failover to the other one for the duration of the outage.

---

From: [email-masked]
Date: Thu, 2 Mar 2017 15:05:09 +0100

Simon,

> I could force the mail service to bind to just one address - but that then means more local config - and it breaks if the latter connection goes down and I need it to failover to the other one for…

You are right. The effect of MHMP is that the host picks the exit circuit by the act of picking the source address.
(See also the work on Source address dependent routing). Fred Baker has also a few drafts on this topic.

In the context of MHMP there is essentially two ways this _could_ work:

1) The network passes "policy" down to the hosts. And trust the hosts to do the right thing (possibly combined with some policing). That may be something PVDs can solve. Or not.
2) You leave it up to the hosts completely aka throw spaghetti on the wall. E.g. with MP-TCP or SCTP, where all exit circuits are used and the transport (or more appropriately named session layer) deals with exit circuit failure, load balances across connections and so on.

This is a tussle between network control versus host control. Neither of which are ready. I used to run IPv6 MHMP at home. Gave up.
ILNP might be your friend. Or just cave and do NPT66. Or use LISP to multi-home via the support of a 3rd party.

Now that answer made you happy I'm sure... ;-)

Cheers,
Ole

---

From: "Eric Vyncke (evyncke)" <[email-masked]>
Date: Thu, 2 Mar 2017 16:28:06 +0000

Simon,

You nail the problem that the draft https://tools.ietf.org/html/draft-bruneau-pvd-00 is trying to address ;-)

Comments are welcome of course (I will send a separate email for those comments in a few minutes).

The other document I was refering to is now expired: https://tools.ietf.org/html/draft-ietf-6man-multi-homed-host-10 

-éric

On 02/03/17 14:45, "v6ops on behalf of Simon Hobson" <[email-masked] on behalf of [email-masked]> wrote:

---

From: Fred Baker <[email-masked]>
Date: Thu, 2 Mar 2017 08:31:19 -0800

> On Mar 2, 2017, at 6:05 AM, [email-masked] wrote:
> 

I'm not the only one :-)

Simon, take a look in rtgwg. There is some work happening for source/destination routing in OSPF, IS-IS, BGP, and I believe Babel, and a problem statement being considered for adoption in rtgwg as we speak. The BGP prototype is operational in CERNET2, I'm told.

https://datatracker.ietf.org/doc/draft-baker-ipv6-isis-dst-src-routing
https://datatracker.ietf.org/doc/draft-bowbakova-rtgwg-enterprise-pa-multihoming
https://datatracker.ietf.org/doc/draft-ietf-ospf-ospfv3-lsa-extend
https://datatracker.ietf.org/doc/draft-ietf-rtgwg-dst-src-routing
https://datatracker.ietf.org/doc/draft-ietf-rtgwg-multihomed-prefix-lfa
https://datatracker.ietf.org/doc/draft-xu-ospf-multi-homing-ipv6
https://datatracker.ietf.org/doc/draft-xu-src-dst-bgp

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 3 Mar 2017 10:29:59 +1300

On 03/03/2017 02:45, Simon Hobson wrote:
> That doesn't really address the issue. 

That would be https://tools.ietf.org/html/rfc8028 and it addresses the *host* side of the issue. For the routing side, see Fred's message  (https://mailarchive.ietf.org/arch/msg/v6ops/mMcqemOcu9U5D7fWyuIS0UNKg4w ).

Work in progress though, not solved yet. However, it avoids the complexity of PVDs.

    Brian

> It's logical that for whichever address the node picks to use as a source address for an outbound connection, it should use an appropriate outbound route - I would guess, logically a router that advertised the matching prefix as one it routes for.
> 
> .

---

From: Tim Chown <[email-masked]>
Date: Thu, 2 Mar 2017 23:57:16 +0000

On 2 Mar 2017, at 16:28, Eric Vyncke (evyncke) <[email-masked]<mailto:[email-masked]>> wrote:

Simon,

I assumed you meant https://tools.ietf.org/html/draft-bowbakova-rtgwg-enterprise-pa-multihoming-01 :)

But the IETF has been kicking this particular football around for at least 15 years now, since before the multi6 WG was born and died… looking at https://datatracker.ietf.org/wg/multi6/documents/ is rather nostalgic!

Tim

-éric

On 02/03/17 14:45, "v6ops on behalf of Simon Hobson" <[email-masked]<mailto:[email-masked]> on behalf of [email-masked]<mailto:[email-masked]>> wrote:

   Eric Vyncke (evyncke) <[email-masked]<mailto:[email-masked]>> wrote:

How timely :-)

   _______________________________________________
   v6ops mailing list
   [email-masked]<mailto:[email-masked]>
   https://www.ietf.org/mailman/listinfo/v6ops

_______________________________________________
v6ops mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/v6ops

---

From: Simon Hobson <[email-masked]>
Date: Fri, 3 Mar 2017 16:53:15 +0000

> https://datatracker.ietf.org/doc/draft-xu-src-dst-bgp

I think you've missed the issue I'm looking to address - on a quick peruse, it looks like only the second of those links addresses source address selection by the host/service. While these various routing protocol extensions may well be important in the larger networks, I believe the vast majority of small to medium business have no need of those - the big problem is not how to route the packets for a given source address(prefix), but how to select the source address(prefix) in the first place.

Taking my own situation at work. I have a single router - it's actually a Debian 7 (Wheezy) virtual machine and already doing source based routing for IPv4 traffic across two egress routes. Extending that to IPv6 is relatively easy - "simply" a matter of configuring the routing rules in much the same way as is currently done for IPv4.
This is echoed at some of our customer sites where they have a "small business" grade router (typically Draytek) supporting multiple WAN interfaces. The Draytek routers have policy based routing - "traffic matching this set of criteria should be routed via that WAN interface".

Where there are multiple routers (eg separate routers provided by each ISP), then my *assumption* would be that the IPv6 stack would forward packets to the router which advertised the prefix containing the IP address the host/service has chosen to use for the outbound connection. In general we would frown upon this setup since it doesn't work well in the IPv4 world - in the general case it means manual configuration on nodes to select the egress router and is inflexible.

---

From: "Russ White" <[email-masked]>
Date: Fri, 3 Mar 2017 13:47:56 -0500

> I think you've missed the issue I'm looking to address - on a quick
peruse, it
> looks like only the second of those links addresses source address
selection
> by the host/service. While these various routing protocol extensions may
> well be important in the larger networks, I believe the vast majority of
small
> to medium business have no need of those - the big problem is not how to
> route the packets for a given source address(prefix), but how to select
the
> source address(prefix) in the first place.

I think there is something in there that does support that use case--part of
this work is around homenet, which should provide the scale down to the
small home office. Of course it requires changes to the host/etc., but I
think it does solve the problem you've described...

:-)

Russ

---

From: Simon Hobson <[email-masked]>
Date: Fri, 3 Mar 2017 19:30:33 +0000

> Now that answer made you happy I'm sure... ;-)

Well actually, it does (in a way) make me happy - or at least a bit less glum. At least I know I'm not alone, and there isn't something obvious that I've missed !

It does seem that, at the moment, there isn't a simple answer to the problem - except perhaps NPT

---

From: Fred Baker <[email-masked]>
Date: Fri, 3 Mar 2017 12:03:03 -0800

> I think you've missed the issue I'm looking to address - on a quick peruse, it looks like only the second of those links addresses source address selection by the host/service. While these various…

In that case, I think I have to refer you to 6man. They have spent an inordinate amount of time trying to tell hosts how to select a source address, and wound up mostly saying "please don't use one you think might be NAT'd, and please don't use 6to4" and otherwise leave the device to select one at random. Yes, one could structure the table in an enterprise deployment. But - who does?

v6ops isn't about the host, it's about the network as seen by its operator. 6man is about the host.

https://tools.ietf.org/html/rfc6724
6724 Default Address Selection for Internet Protocol Version 6 (IPv6). D.
     Thaler, Ed., R. Draves, A. Matsumoto, T. Chown. September 2012.
     (Format: TXT=74407 bytes) (Obsoletes RFC3484) (Status: PROPOSED
     STANDARD) (DOI: 10.17487/RFC6724)

And of course it's predecessor.

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 4 Mar 2017 11:02:24 +1300

> And of course it's predecessor.

In my mind, for a smallish site, where source-address based routing is
not really a factor except at the exit router, that is also helped
by https://tools.ietf.org/html/rfc8028, which does make a specific
recommendation about RFC6724 Rule 5.5.

Setting up an appropriate source-routing policy in the exit router
certainly seems preferable to NPTv6.

Also see https://tools.ietf.org/html/rfc7157 of course.

(Most of that work is aimed at enterprise networks. I agree that the
Homenet solution should be the best one for SOHO networks, once it
becomes off-the-shelf.)

    Brian

---

From: Gert Doering <[email-masked]>
Date: Mon, 6 Mar 2017 20:49:57 +0100

Hi,

On Fri, Mar 03, 2017 at 04:53:15PM +0000, Simon Hobson wrote:
> Where there are multiple routers (eg separate routers provided
> by each ISP), then my *assumption* would be that the IPv6 stack
> would forward packets to the router which advertised the prefix
> containing the IP address the host/service has chosen to use for
> the outbound connection.

That only works if these routers are actually directly adjacent to
the host making the decision.  In the more generalized homenet model,
routers could be meshed in any ways, and the routers injecting a 
/48 into "the homenet mesh" could all be talking through an intermediate
router to the host - so the host would see a RA with (say) 3x /64 coming
from the same router.

But this is easily solved by source-dependent routing - "forward packets
with a source address from ISP A out via the line to ISP A, and so on
for ISP B, C, ...".

The source address selection part is much harder, and to my knowlege,
still mostly unsolved.  6724 has a list of rules that are not really
helping your use case ("multiple ISPs that have different characteristics,
and some of them might not even be working right in this moment") - 
especially, source-address *failover* is not something I've seen work
("I tried source A, and it did not work, so maybe ISP A has a problem
reaching target Z.  Let's try source address B, via ISP B -> Z").

> In general we would frown upon this setup since it doesn't work well in the IPv4 world - in the general case it means manual configuration on nodes to select the egress router and is inflexible.

The homenet architecture puts this nicely on the routers, and that part
really works quite well ("built a test setup, liked the result").

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: Simon Hobson <[email-masked]>
Date: Tue, 7 Mar 2017 08:07:59 +0000

> with a source address from ISP A out via the line to ISP A, and so on

Indeed, and I already have that in place for IPv4 at work (but using NAT and RFC1918 addressing). I've limited IPv6 running as management don't seem to have realised it exists yet !

> still mostly unsolved.

That makes sense - the hard bit is still unsolved :-(

>  6724 has a list of rules that are not really

Well that's an interesting approach.
I suppose that if a router has rules about what traffic is allowed via each upstream connection, then it could reject attempts on the other connection(s). So that becomes "tried source address A, got ICMP <something> back, lets try B" which given the speed of having an ICMP packet back vs the connection timing out might not have too much overhead/delay attached to it.
But for the time being, it looks very much like it's going to require manual setting of rules for each process (eg binding services to specific addresses) with all the implications that has for handling changes in connectivity.

---

From: Tim Chown <[email-masked]>
Date: Tue, 7 Mar 2017 14:59:49 +0000

On 6 Mar 2017, at 19:49, Gert Doering <[email-masked]<mailto:[email-masked]>> wrote:

Hi,

Yes, and I recall it was demonstrated at a hackathon several IETFs ago, with Markus Stenberg’s code, largely, but that played shenanigans with OSPF, and things have moved on a little in homenet since then :)

Otherwise, we’re back at https://tools.ietf.org/html/draft-bowbakova-rtgwg-enterprise-pa-multihoming-01

Tim