Subject: [Rift] quick review of RIFT 04

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 27 Feb 2018 13:30:42 +0000

Hello Tony :

Please find first read comments below :

1)      I think that the abstract is a bit heavy and the intro quite thin. The second paragraph in particular is critical to understand the following sections smoothly and yet is a bit cryptic at first read. I'd really like to see an image in intro showing a more abstract view of the fat network than section 2.2, with "Link state" topological awareness sent north and "distance vector" aggregations sent south, and text showing how the spine gets a clear view of the network whereas the leaves get a very rough understanding of what's above.

2)      I'd also like to see an early discussion on the default route. Is it used? Up? E/W?.

3)      The image could also depict a simple case of aggregation/disaggregation to clarify what's coming next.

4)      Section 3 could move to annex; it is really interesting but not normative.

5)      Pointing North and south in Section 4 is still hard to fathom. Do you point north for sending LSAs or are the LS routes pointing north? I think I know the answer but the text needs to indicate that without a doubt. Like I suggested, even better would be in intro.

6)      4.2.1 a note that RPL uses ICMP code 155 and ICMP types to differentiate the messages. It was pointed to us that it was a mistake to consume ICMP due to shortage; OTOH, RIFT and RPL being is a same family, It could be possible to use additional types of ICMP 155 for RIFT.

7)      In the following, 0 seems overloaded. Note that RPL uses a Rank of 1 for the "spine", which is allows to build a virtual "spine of spines", or brain, at level 0. It also has an "infinite" Rank for leaves, which allows to poison the DV.

"   Unless Section 4.2.9<https://tools.ietf.org/html/draft-przygienda-rift-04#section-4.2.9> is used, each node is provisioned with the level

   at which it is operating and its PoD or otherwise a default level and

   PoD of zero are assumed, meaning that leafs do not need to be

   configured with a level (or even PoD).  Nodes in the spine are

   configured with a PoD of zero."

8)      4.2.3.1 indicates that some TIE is propagated south. Which is surprising in a DV world. The reader (me) wonders, is it really DV or is it more like fisheye?

9)      Table 1 is hard to read, because it is dense and rows are not well-delineated; using style=all | full in texttable may help; same goes for table 2 and 3 though not as badly.

10)  Note that TIE, LIE (and please do not use DIE!) match English words, and though it is funny it will make a google search harder in the future. I like S-TIE and N-TIE for that matter. An alternate would be a name that describe what they are as opposed to the direction.

11)  4.2.3.1: IPv6 ND uses a Hop Limit of 255 and instruct to drop packets that do not have such a TTL. It's a trick, but maybe useful to detect wrongdoers. The rule is to drop messages received with a HL that is lower than 255.

12)  4.2.3.4: NTIE do not really need to be flooded if (as I understand); a node only needs to talk north to its parents in the POD. RPL uses N*unicast because of unreliable radios. We could devise a mcast group that derives from the POD Nb and the depth.

13)  4.2.3.7 need to define "overloaded" which is introduced here first; could that lead to oscillations?

14)  We discussed sibling routing in RPL a lot. Some of us wanted it in the final protocol but it was filtered out to keep things at their simplest. Point being, I think that the rule below is too harsh. One simple rule could be that can use a sibling route as long as the sibling does not use a sibling again. I ended up generalizing this into the concept of ARCs, which enables structures that one can "walk" (E or W) to find an "exit" without progressing from the metric standpoint, whereas progress towards the destination is done between such structures (N or S).

"This rule forms a "one-hop default route split-horizon" and prevents

   looping over default routes while allowing for "one-hop protection"

   of nodes that lost all northbound adjacencies"

15)  About "East-West Forwarding Within a Level": I  understand that the ring protection is the split horizon technique, correct? I also understand that E/W reachability is not forwarded E/W, so a node only sees what its direct E/W neighbors see northwards, correct? In terms of preference, I understand that south wins, then north regardless of longest match, then E/W as last resort for packets that are ultimately northbound, correct? This seems to indicate that RIFT really maintains 3 RIBs and looks them in order, S, N and then E/W till a solution is found. Arguably S vs. N could be done in a same RIB with longest match. E/W RIB could only be used for packets coming from a Southbound interface.

Cheers

Pascal

---

From: Antoni Przygienda <[email-masked]>
Date: Tue, 27 Feb 2018 15:22:39 +0000

Hey Pascal, many thanks for the drill down. As I see this is super dense and coming from someone who was wandering some of the ground RIFT is putting stakes in already ;-) I need to digest that for a bit so give me a day or two before I answer … And I won’t deny that the RPL talk in the area plenary (?) was one of the things that made me think about DC fabric routing less in traditional IGP ways though I admit that the talk we had in Hawaii about ARCs largely went over my head, maybe due to late hour and couple colorful drinks in the system ;-)

This comes opportune since I have on my burner another rev with more surprises coming from people that run such networks and looked @ RIFT closely ;-)  So let’s go over your stuff and incorporate what falls out before IETF cut-off …

thanks

--- tony

<[email-masked]<mailto:[email-masked]>> spake:

Hello Tony :

1)       I think that the abstract is a bit heavy and the intro quite thin. The second paragraph in particular is critical to understand the following sections smoothly and yet is a bit cryptic at first read. I’d really like to see an image in intro showing a more abstract view of the fat network than section 2.2, with “Link state” topological awareness sent north and “distance vector” aggregations sent south, and text showing how the spine gets a clear view of the network whereas the leaves get a very rough understanding of what’s above.

2)       I’d also like to see an early discussion on the default route. Is it used? Up? E/W?.

3)       The image could also depict a simple case of aggregation/disaggregation to clarify what’s coming next.

7)       In the following, 0 seems overloaded. Note that RPL uses a Rank of 1 for the “spine”, which is allows to build a virtual “spine of spines”, or brain, at level 0. It also has an “infinite” Rank for leaves, which allows to poison the DV.

“   Unless Section 4.2.9<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dprzygienda-2Drift-2D04-23section-2D4.2.9&d=DwMFAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=maKXfKzgRTpiLitqHnJiww&m=ACdtv3rGVGoMIMa7cXNpGEIpriUROA6hZ9y51APRzqo&s=Z8UnKkb5PTiS5EzqzbDm2V2AQFHa6tN7vz3eZEqGIyM&e=> is used, each node is provisioned with the level

   configured with a PoD of zero.”

11)   4.2.3.1: IPv6 ND uses a Hop Limit of 255 and instruct to drop packets that do not have such a TTL. It’s a trick, but maybe useful to detect wrongdoers. The rule is to drop messages received with a HL that is lower than 255.

13)   4.2.3.7 need to define “overloaded” which is introduced here first; could that lead to oscillations?

14)   We discussed sibling routing in RPL a lot. Some of us wanted it in the final protocol but it was filtered out to keep things at their simplest. Point being, I think that the rule below is too harsh. One simple rule could be that can use a sibling route as long as the sibling does not use a sibling again. I ended up generalizing this into the concept of ARCs, which enables structures that one can “walk” (E or W) to find an “exit” without progressing from the metric standpoint, whereas progress towards the destination is done between such structures (N or S).

“This rule forms a "one-hop default route split-horizon" and prevents

   of nodes that lost all northbound adjacencies”

15)   About “East-West Forwarding Within a Level”: I  understand that the ring protection is the split horizon technique, correct? I also understand that E/W reachability is not forwarded E/W, so a node only sees what its direct E/W neighbors see northwards, correct? In terms of preference, I understand that south wins, then north regardless of longest match, then E/W as last resort for packets that are ultimately northbound, correct? This seems to indicate that RIFT really maintains 3 RIBs and looks them in order, S, N and then E/W till a solution is found. Arguably S vs. N could be done in a same RIB with longest match. E/W RIB could only be used for packets coming from a Southbound interface.

Cheers

Pascal

---

From: Antoni Przygienda <[email-masked]>
Date: Thu, 1 Mar 2018 06:04:10 +0000

Getting to it ;-)

--- tony

Hello Tony :

I expanded 2nd paragraph in the intro and put some “first picture” into it. I hope that helps.

Introducting E/W early tends to muddle the picture, that’s why it’s pushed further back in the draft. First, fat-trees & CLOS do NOT have horizontal links in normal scenarios. Then, introducting E/W link in high capacity Clos is a mixed blessing @ best, RIFT uses it for backup purposes only (unless PGP manually force different behavior) for the simple reason that trying to load-balance through them can easily generate delay asymmetries, bandwidth bottlenecks and lead then to losses. Ultimately, once we allow bi-directional E/W links in normal condition we end up with a very complex protocol that starts to move closely to flat IGP and is not loop-free anymore. Ongoing discussion really but the current trade-off seems to hold.

ASCII art is not well suited to complex pictures ;-) and I prefer to write in a way where concepts are developed by guiding the reader along the problems that emerge one after the next rather than trying to pile e’thjing in @ once. I allured in the introduction to disaggregation though.

I actually expect it may be even split out as a separate document but it is very purposefully put where it is. RIFT is a drastic departures of current dynamic routing (leaving RPL aside) and hence needs the “why” even more than “how” …

I hope the intro figure clarifies it somewhat. Expanded intro to 4 a bit as well.

?

Rewrote, cleaned up language to “undefined” PoD

Fisheye didn’t enter my dictionary of routing yet. Yes, it’s DV but “flooded over TIEs”, each time I try to explain that I create more confusion. It only becomes clear what that means in operation.

done

The S-X and N-X is very consciously chosen since the protocol is highly symmetric and the distributed information has an algebra that starts to emerge the deeper you understand it. The TIE/LIE/TIDE choices are very consciously rooted in the way we easily learn and remember things but this is not the forum to dwelve into memory organization structures and psychological foundations of knowledge acquisition ;-)

Google is Google, they’ll bring up what they bring up ;-)

Yes, never been a big friend of that. It has as many advantages as disadvantages, biggest advantages being the “security”, biggest disadvantage that we’re talking multicast on LIEs anyway which is local scope & in case of TIE one mistaken address will blast flooding over 255 hops possibly. So I prefer 1 right now.

Unfortunately, N-TIEs do need to be flooded all the way to the top, otherwise the top of the fabric cannot decide whether to go “left down” or “right down” when heading towards a destination since the “buck stops” @ the top ;-)  I refer to Table 2 “Flooding Scopes” which explains what floods where & how far.

Footnote for the deeper thinkers here, skip safely: Flooding scopes are an “algebra” over TIEs but they are just a phenomenon. The thing-in-itself are the “epistemological scopes of knowledge synchronization necessary on the fabric”. Epistomology acquired quite a status in current distributed systems theory and I lean on it heavily to shape RIFT as a protocol but I leave that under the hood mostly so that’s something we can talk over beer ;-) Or in simpler terms I apply strictly a “when and need to know” principle in designing the whole thing.

Oh, sorry, that’s a well known term in link-state ISIS, means “overload bit set”. I’ll add to glossary …

That’s something too complex for email. London.

So, almost right. No, we maintain a single RIB where each route is tie-broken based on `enum RouteType` and then normal LPM used … The E/W discussion is f2f again …

Will release up withj more stuff in a day or two ..

Cheers

Pascal