Subject: [pim] pimd and high availability/performance

From: Vincent Bernat <[email-masked]>
Date: Wed, 05 Jan 2011 19:38:55 +0100

Hi!

I would like  to use pimd in network where  routers share three distinct
networks for  redundancy and performance  purpose. For unicast,  OSPF is
running  allowing routers  to  have three  available  routes to  contact
another router. Moreover,  each router has a backup  that will take over
its role in case of failure.

#v+
 R 1 R 2 R 3 R 4 R 5 R 6
 ||| ||| ||| ||| ||| |||
 ||+-||+-||+-||+-||+-||+------ 10.0.0.0/24
 |+--|+--|+--|+--|+--|+------- 10.0.1.0/24
 +---+---+---+---+---+-------- 10.0.2.0/24
#v-

Thanks to OSPF, the routing table of R1 contains things like this:

192.168.0.0/24 proto zebra metric 20
   nexthop via 10.0.0.3 dev eth0 weight 1
   nexthop via 10.0.1.3 dev eth1 weight 1
   nexthop via 10.0.2.3 dev eth2 weight 1

10.0.0.3, 10.0.1.3 and 10.0.2.3 are IP for R3.

With this setup, the use of ECMP (Equal Cost Multipath), we can ensure 2
GBps of output bandwidth even in case of one link failure.

If a link  fails, OSPF is pretty  fast to remove the fautive  hop of the
route (less than 5 seconds).

If the designated router fails, OSPF  is also pretty fast to promote the
backup designated router.

If a router  fails, its backup starts to advertise  the same routes. For
example, if R3 fails, its backup router R4 will advertise 192.168.0.0/24
network the same way than R3. This takes less than 5 seconds to happen.

I have configured pimd such that each  router can be CRP and BSR on each
interface (eth0,  eth1, eth2). I have  run some small  experiments and I
have noticed three things :
 - there is no ECMP
 - a link failure is solved in less than 5 seconds
 - if the RP fails, there is a downtime of more than 1 minute
 - if a backup router takes over, the downtime is also more than 1 minute

My questions are :
 - is there a way to have ECMP multicast with pimd and Linux?
 - is there a way to speed up the recovery when the RP is lost?
 - how to reduce the take-over time?

For  the last point,  here is  how we  work with  OSPF.  Each  couple of
routers   are  responsible   for  a   set  of   networks   (for  example
192.168.0.0/24 is a network behind R3 and R4). With the help of VRRP, R3
and R4 are sharing a VIP  (for example 192.168.0.1).  When R3 is master,
it has the VIP and R4 has  no IP for this network. Therefore, only R3 is
advertising this network  with OSPF. When a take-over  happens, R4 takes
the VIP and immediatly, OSPF starts to advertise the network.

The weakness in my setup of pimd is that I am not able to reproduce this
since pimd  will refuse to do  anything with an interface  without an IP
and  does   not  notice  when  an   interface  gains  or   loses  an  IP
address. Therefore,  when a take-over  happens, I reload pimd.  This may
explain the important time needed to recover the situation.

I don't want  to assign IP for 192.168.0.0/24 network to  both R3 and R4
because both of them will start advertising the network through OSPF and
therefore,  IP  packets  may  come  through R4  and  come  back  through
R3. Since R3 and R4 are also stateful firewalls, this will not work.

Any hint  to reduce down time with  pimd in case of  topology changes is
welcome.
-- 
I WILL NOT TRADE PANTS WITH OTHERS
I WILL NOT TRADE PANTS WITH OTHERS
I WILL NOT TRADE PANTS WITH OTHERS
-+- Bart Simpson on chalkboard in episode 7F05