Subject: [homenet] Why configuration and routing are separate

From: Juliusz Chroboczek <[email-masked]>
Date: Sat, 23 Jul 2016 01:28:20 +0200

Speaking with people at this week's IETF meeting, I've realised that some
are confused about why configuration and routing are implemented by
different protocols in Homenet.  Contrary to what might seem, this is not
some sort of weird political compromise, but a conscious technical
decision.

Routing and configuration have very different dynamics -- configuration is
much more static.  If a link becomes lossy, the prefix it is assigned by
the configuration protocol does not change, but its routing metric must
increase and be propagated across the metric.  If a router crashes, it
doesn't harm if the prefixes it assigned to its interfaces remain marked
as used for a few minutes; the routing protocol, on the other hand, must
be able to reroute within a fraction of a second.

On the other hand, the configuration protocol must be able to carry
a fairly large amount of data (on the order of 1kB per node in the current
implementation of HNCP), while the routing protocol does not -- it only
carries a single metric per address, or, in the case of link state,
a single metric per interface.

Because of that, it is best to implement routing and configuration in
different protocols, each of which is carefully designed for its intended
usage.  What is more, it avoids the need to include large amounts of
non-routing code in a routing daemon, something that the routing people,
who have spent years fine-tuning their implementation, are not likely to
take in a friendly manner (and, IMHO, rightly so).

Please let me know if you're still unconvinced.  I love arguing.

Regards,

-- Juliusz

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 23 Jul 2016 17:48:51 +1200

> some sort of weird political compromise, but a conscious technical

...

> Please let me know if you're still unconvinced.  I love arguing.

You are 100% correct. They should be separately pluggable, in fact,
in case either of the current choices proves to be a mistake later on.

(Some people may recall that I used homenet as a use case in the UCAN
BOF that preceded the Anima working group. I wish I had articulated
it as clearly as Juliusz has done in the text deleted above. It will
be an interesting exercise, once Anima has done its work, to check that
it would in fact work for a homenet, although its style would be very
different from HNCP.)

   Brian

---

From: Mikael Abrahamsson <[email-masked]>
Date: Sat, 23 Jul 2016 08:30:49 +0200 (CEST)

> Please let me know if you're still unconvinced.  I love arguing.

Well, in my testing I got the feeling (hard to tell since it was really 
hard to get a comprehensive picture of what was going on over time), that 
sometimes HNCP lost its connection to other HNCP nodes on the lan, while 
babel was still working, and the other way around, babel went down and 
HNCP was up.

Having these two protocols knowing nothing about each other and each 
others' state is potential source of problems.

I'm trying to think of another case where this has caused problem. A 
non-perfect example is MPLS requiring LDP to be up before the IGP starts 
trying to use the LSP switch path. Thus the "ldp igp sync" feature that 
came in RFC5443 which was quite a lot later in time than when RFC3031 
first defined MPLS architecture. This feature came around after 
operational experience with the protocols and that them not knowing 
anything about each other caused operational problems and outages.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Henning Rogge <[email-masked]>
Date: Sat, 23 Jul 2016 10:50:48 +0200

On Sat, Jul 23, 2016 at 8:30 AM, Mikael Abrahamsson <[email-masked]> wrote:
> Well, in my testing I got the feeling (hard to tell since it was really hard
> to get a comprehensive picture of what was going on over time), that
> babel was still working, and the other way around, babel went down and HNCP
> was up.

I wonder if HNCP could have a local "port" where a routing protocol
could deliver the information that a certain linklocal neighbor
address is still reachable.

HNCP would still do its own neighbor detection, but it could take
advantage of the (regularly happening) checks of the links of the
routing.

Henning Rogge

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sat, 23 Jul 2016 11:41:16 +0200

> babel was still working, and the other way around, babel went down and

Agreed, that's the price of having two distinct link-sensing mechanisms.

> others' state is potential source of problems.

Things are meant to be so the network will survive desynchronisation
between the two protocols:

  - HNCP up, Babel down => numbered link not used for transit;
  - HNCP down, Babel up => unnumbered link used for transit.

If either of these causes problems in your network, then something is
wrong, and I'm interested in knowing more.

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sat, 23 Jul 2016 14:39:10 +0200

> others' state is potential source of problems.

I've thought about it some more (warm showers increase the flow of blood
to my brain), and I've realised that this is an important point, and one
that is close to my research interests.  So please let me ramble some more.

Given a distributed system (such as, say, a network of IS-IS nodes, or
a network of Kademlia nodes, or a network of HNCP + Babel nodes), in
transitory situtations the different states will temporarily be
inconsistent.  For example, the LS databases of OSPF nodes will
temporarily desynchronise, a node will leave the Kademlia network without
notifying its neighbours, or HNCP and Babel will have different ideas
about an adjacency being up or down.  I believe this is a law of nature --
but I haven't been able to find the theorem that says that.

(Aside: I'm positive there's a theorem that says that distributed consensus
cannot be achieved by a symmetric, deterministic algorithm, but
disappointingly enough this doesn't apply to networking protocols which use
random backoff (non-deterministic) or break ties by picking the larger
node-id (broken symmetry).  End of aside.)

It is generally considered ungentlemanly to attempt to change the laws of
nature, so people have taken two approaches to work around the problem:

  (1) minimise the time window during which nodes are desynchronised;
  (2) minimise the consequences of desynchronisation.

To give an example of approach (1), link-state routing protocols have been
carefully engineered to reflood in a timely and reliable manner -- this
aims to minimise the time during which the network is pushing packets in
the wrong direction.  Examples of (2) include Kademlia, which is designed
to survive large numbers of broken nodes (our measurements indicate that
40 to 60% of nodes in the BitTorrent DHT are broken), and protocols such
as EIGRP, DSDV or Babel, which are designed to keep pushing packets in the
right direction following loop-free paths even during reconvergence.

Note that the two approaches are not exclusive -- just because Kademlia
can deal with broken nodes doesn't mean we should be encouraging
brokenness, and just because Babel is loop-avoiding doesn't mean we should
be slowing down its convergence.  (I am not aware of any work about
transient behaviour of link-state protocols, perhaps the OLSR community
has done something related?)

I don't think it has ever been stated explicitly, but my view is that (2)
is what we are doing with HNCP + Babel: allowing the two protocols to
transiently have different ideas about adjacencies, and making sure that
nothing important breaks when that happens.  Of course, just because
nothing breaks doesn't mean that it's a desirable situation, so adding
(optional) mechanisms to minimise the likelihood of this happening is
something we should be considering.  I'm not sure that Henning's
suggestion is worth the implementation cost, however.

-- Juliusz