Subject: Re: [Lsr] Open issues with Dynamic Flooding

From: [email-masked]
Date: Mon, 1 Apr 2019 10:37:32 -0700

Hi all,

I hope that everyone had a safe and uneventful trip home from Prague and that no one else had the seat right in front of the screaming baby.  ;-)

I would like to re-open the discussion on the mailing list. Based on the off-line discussions that I had with folks, I believe that we’re leaning towards including the LANs in the signaling and rate limiting link addition during repair.

Dissent? Discussion?

Tony

> On Mar 4, 2019, at 9:54 AM, [email-masked] wrote:
> 
> 
> Hello,
> 
> There are still two issues that need to be discussed and I was hoping that we could make progress on the mailing list before Prague.
> 
> 1) Temporary additions to the flooding topology
> 
>    There are several cases where we would like to make temporary additions to the flooding topology: repairing a partition of the flooding topology or adding a node to the base topology for the first time. We can:
> 
>    (a) Temporarily add all of the links that would appear to remedy the partition. This has the advantage that it is very likely to heal the partition and will do so in the minimal amount of convergence time.
> 
>    (b) For each node adjacent to the partition, add no more than a single link across the partition.  If that does not repair the partition in a while (LSP propagation time + SPF time), then add another link.  
>         Iterate as necessary. This has the advantage that it minimizes the risk of creating a cascade failure.
> 
> 2) Inclusion of pseduonodes in the System IDs TLV
> 
>    In the general case, a topology can include LANs. If a LAN is in parallel with a P2P link, the Area Leader cannot currently distinguish between the two links. This can be of importance if there are other 
>    systems also on the LAN that should be using their LAN interface for flooding.
> 
>    We propose to change the System IDs TLV to include a pseudo-node ID as well as the system ID.  It would also make sense to rename the TLV to be the “IS-IS Area Node IDs TLV”.
> 
>    Behaviorally, we should add a requirement that if the Area Leader includes a pseudonode in the flooding topology, then all systems with an adjacency on that LAN should use the LAN as part of the 
>    flooding topology, whether or not they are explicitly listed as adjacent to the LAN in the Flooding Path TLV.
> 
> Thoughts? Comments? Flames?
> 
> Regards,
> Tony
>

---

From: Huaimo Chen <[email-masked]>
Date: Tue, 2 Apr 2019 18:10:11 +0000

Hi Tony,

    Can you give some details about: what is the rate limiting link addition and how does it (the rate limiting link addition) fix or help fix the flooding topology (FT) split when multiple failures occur on FT?

Best Regards,
Huaimo
-----Original Message-----
From: Lsr [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Monday, April 1, 2019 1:38 PM
To: [email-masked]
Subject: Re: [Lsr] Open issues with Dynamic Flooding

Hi all,

Dissent? Discussion?

Tony

> Thoughts? Comments? Flames?

_______________________________________________
Lsr mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/lsr

---

From: [email-masked]
Date: Tue, 2 Apr 2019 14:33:55 -0700

Hi Huaimo,

> Can you give some details about: what is the rate limiting link addition and how does it (the rate limiting link addition) fix or help fix the flooding topology (FT) split when multiple failures oc…

The details of the wording will have to wait until someone (probably Peter) drafts them. In general, what we do in such cases is to leave the implementation details to the respective implementations.

As we’ve discussed, when there is a partition of the FT (or equivalently, new nodes that are not part of the FT), we need to add links temporarily to the FT to restore flooding. We agree that we want to add a link when one (or more) of the parties appears to not be on the FT.  If we happen to pick the right links to add temporarily, this should repair the partition.

Where we have had a discussion was about how many links to add.  As I’ve described, some people feel that adding links aggressively is the best course of action, because convergence time is an issue.

Others feel that adding too many links too quickly may lead to excessive flooding and a possible cascade failure.

Rate limiting is something of a compromise between these two extremes.

Yes, I freely admit that it is a weasel-wording approach to a compromise.  It basically implies that anyone can do anything that they want.  Until we clearly know more, it’s the best that we can do.

Call it an agreement to disagree. :-)

Regards,
Tony

---

From: [email-masked]
Date: Mon, 8 Apr 2019 07:27:25 -0700

Hi all,

It’s been another week and we’ve had a few more very interesting conversations, but we seem to have not moved very far.

Have we converged?

Tony

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Tue, 9 Apr 2019 15:09:02 +0000

Tony -

Here is my take.

Regarding Issue #2 below, we had a healthy thread on this since Prague and I believe have consensus that we WILL support LANs in the encoding of the flooding topology (centralized mode). Authors need to agree on changes to the draft which we will take offline and then publish an update.

Regarding Issue #1 below, we did have a thread on this BEFORE Prague and seemed to reach consensus on:

<snip>
Let me propose that we add something to sections 6.7.5, 6.7.9, and 6.7.11 like:

Addition of temporary flooding should be done with caution, as the addition of excessive connectivity to the flooding topology may trigger unwanted behavior. Routers SHOULD add temporary flooding in a rate limited manner, if not configured otherwise.

<end snip>

(See attached email)

Again, authors need to address this in the next draft revision but I believe we have agreement in principle.

So I think we can consider these matters resolved - pending WG feedback on the updated draft whenever it becomes available.

   Les

> From: Lsr <[email-masked]> On Behalf Of [email-masked]
> Sent: Monday, April 08, 2019 7:27 AM
> To: [email-masked]
> Subject: [Lsr] Fwd: Open issues with Dynamic Flooding
> 
> 
> Hi all,
> 
> It’s been another week and we’ve had a few more very interesting
> conversations, but we seem to have not moved very far.
> 
> Have we converged?
> 
> Tony
> 
> 
> 
> > Hi all,
> >
> > I hope that everyone had a safe and uneventful trip home from Prague and
> that no one else had the seat right in front of the screaming baby.  ;-)
> >
> > I would like to re-open the discussion on the mailing list. Based on the off-
> line discussions that I had with folks, I believe that we’re leaning towards
> including the LANs in the signaling and rate limiting link addition during repair.
> >
> > Dissent? Discussion?
> >
> > Tony
> >
> >
> >> There are still two issues that need to be discussed and I was hoping that
> we could make progress on the mailing list before Prague.
> >>
> >>   There are several cases where we would like to make temporary
> additions to the flooding topology: repairing a partition of the flooding
> topology or adding a node to the base topology for the first time. We can:
> >>
> >>   (a) Temporarily add all of the links that would appear to remedy the
> partition. This has the advantage that it is very likely to heal the partition and
> will do so in the minimal amount of convergence time.
> >>
> >>   (b) For each node adjacent to the partition, add no more than a single
> link across the partition.  If that does not repair the partition in a while (LSP
> propagation time + SPF time), then add another link.
> >>   In the general case, a topology can include LANs. If a LAN is in parallel
> with a P2P link, the Area Leader cannot currently distinguish between the
> two links. This can be of importance if there are other
> >>   We propose to change the System IDs TLV to include a pseudo-node ID
> as well as the system ID.  It would also make sense to rename the TLV to be
> the “IS-IS Area Node IDs TLV”.
> >>
> >>   Behaviorally, we should add a requirement that if the Area Leader
> includes a pseudonode in the flooding topology, then all systems with an
> adjacency on that LAN should use the LAN as part of the

Subject: Re: [Lsr] Open issues with Dynamic Flooding
From: "[email-masked]" <[email-masked]>
To: "Peter Psenak (ppsenak)" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Date: Tue, 5 Mar 2019 21:31:22 +0000
List-Id: Link State Routing Working Group <lsr.ietf.org>

>> I agree that optimal is probably unknowable. The question then is what
>> do we say in the document?  How about something about rate limiting?
> 
> yes, something of that nature, with possible config option.

Let me propose that we add something to sections 6.7.5, 6.7.9, and 6.7.11 like:

Addition of temporary flooding should be done with caution, as the addition of excessive connectivity to the flooding topology may trigger unwanted behavior. Routers SHOULD add temporary flooding in a rate limited manner, if not configured otherwise.

> the trick is that "all" nodes would comply, where we may only need one/subset to do...

This is the addition of one link, and in particular the onus is really on the DIS to drive synchronization.  Per your above arguments, I’m comfortable with this.

Tony

---

From: Huaimo Chen <[email-masked]>
Date: Tue, 9 Apr 2019 15:47:03 +0000

Hi Les,

    For "add temporary flooding in a rate limited manner", can you give some details about how does the rate limit manner work for fixing a FT split? how does each node get the rate limit? Will every node add temporary flooding on a given number of links independently? If so, there are lots of links to be added into the FT temporarily for fixing the FT split. This may cause some issues.
    In another thread "[Lsr] Min Links for Multiple Failures on Flooding Topology", there is a solution for fixing the FT split using almost minimum number of links. 

Best Regards,
Huaimo
-----Original Message-----
From: Lsr [mailto:[email-masked]] On Behalf Of Les Ginsberg (ginsberg)
Sent: Tuesday, April 9, 2019 11:09 AM
To: [email-masked]; [email-masked]
Subject: Re: [Lsr] Fwd: Open issues with Dynamic Flooding

Tony -

Here is my take.

<snip>

<end snip>

(See attached email)

   Les

> line discussions that I had with folks, I believe that we’re leaning 
> towards including the LANs in the signaling and rate limiting link addition during repair.
> >
> > Dissent? Discussion?
> >
> > Tony
> >
> >
> additions to the flooding topology: repairing a partition of the 
> flooding topology or adding a node to the base topology for the first time. We can:
> >>
> partition. This has the advantage that it is very likely to heal the 
> partition and will do so in the minimal amount of convergence time.
> >>
> link across the partition.  If that does not repair the partition in a 
> while (LSP propagation time + SPF time), then add another link.
> >>        Iterate as necessary. This has the advantage that it 
> >> minimizes the risk
> of creating a cascade failure.
> >>
> with a P2P link, the Area Leader cannot currently distinguish between 
> the two links. This can be of importance if there are other
> >>   systems also on the LAN that should be using their LAN interface 
> >> for
> flooding.
> >>
> as well as the system ID.  It would also make sense to rename the TLV 
> to be the “IS-IS Area Node IDs TLV”.
> >>
> includes a pseudonode in the flooding topology, then all systems with 
> an adjacency on that LAN should use the LAN as part of the
> >>   flooding topology, whether or not they are explicitly listed as 
> >> adjacent to
> the LAN in the Flooding Path TLV.
> >>

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Tue, 9 Apr 2019 15:50:45 +0000

Huaimo -

I am aware of the other thread and the discussion you are having with Tony.
My reading of it is that you have not yet defined a deterministic solution to this problem - you have only defined the goal. If you do define a deterministic solution, that would be most welcome and we can then incorporate it in the Dynamic Flooding draft.

Please continue the thread w Tony.

Thanx.

   Les

> From: Huaimo Chen <[email-masked]>
> Sent: Tuesday, April 09, 2019 8:47 AM
> To: Les Ginsberg (ginsberg) <[email-masked]>; [email-masked];
> [email-masked]
> Subject: RE: [Lsr] Fwd: Open issues with Dynamic Flooding
> 
> Hi Les,
> 
>     For "add temporary flooding in a rate limited manner", can you give some
> details about how does the rate limit manner work for fixing a FT split? how
> does each node get the rate limit? Will every node add temporary flooding
> on a given number of links independently? If so, there are lots of links to be
> added into the FT temporarily for fixing the FT split. This may cause some
> issues.
>     In another thread "[Lsr] Min Links for Multiple Failures on Flooding
> Topology", there is a solution for fixing the FT split using almost minimum
> number of links.
> 
> Best Regards,
> Huaimo
> Regarding Issue #2 below, we had a healthy thread on this since Prague and I
> believe have consensus that we WILL support LANs in the encoding of the
> flooding topology (centralized mode). Authors need to agree on changes to
> the draft which we will take offline and then publish an update.
> 
> Addition of temporary flooding should be done with caution, as the addition
> of excessive connectivity to the flooding topology may trigger unwanted
> behavior. Routers SHOULD add temporary flooding in a rate limited manner,
> if not configured otherwise.
> 
> <end snip>
> 
> (See attached email)
> 
> So I think we can consider these matters resolved - pending WG feedback on
> the updated draft whenever it becomes available.
> 
>    Les
> 
>

---

From: [email-masked]
Date: Tue, 9 Apr 2019 09:12:49 -0700

Hi Huaimo,

>    For "add temporary flooding in a rate limited manner", can you give some details about how does the rate limit manner work for fixing a FT split?

Nodes that have adjacencies that appear to be crossing the FT partition can enable temporary flooding on that circuit. This will hopefully repair the partition.  If not, the node waits awhile, and then adds another link to the FT.  Iterate until healed.

> how does each node get the rate limit?

Rate limiting causes each node to do so ’slowly’, where the exact values for the rate limiting are implementation specific.

> Will every node add temporary flooding on a given number of links independently?

No, not every node.  Only those that have an adjacency with a node that appears to not be on the FT.

> If so, there are lots of links to be added into the FT temporarily for fixing the FT split.

Yes, because this is a distributed algorithm without any centralized coordination, you can easily imagine circumstances where there are many links that could repair the partition and thus many temporary additions could be made. This is why some feel that rate limiting is wise.

> This may cause some issues.

This may be an understatement. :-)

Tony

---

From: Huaimo Chen <[email-masked]>
Date: Tue, 9 Apr 2019 16:13:32 +0000

Hi Les,

    Thanks much.
    I have defined a deterministic solution.

Best Regards,
Huaimo
-----Original Message-----
From: Les Ginsberg (ginsberg) [mailto:[email-masked]] 
Sent: Tuesday, April 9, 2019 11:51 AM
To: Huaimo Chen <[email-masked]>; [email-masked]; [email-masked]
Subject: RE: [Lsr] Fwd: Open issues with Dynamic Flooding

Huaimo -

Thanx.

   Les

>     For "add temporary flooding in a rate limited manner", can you 
> give some details about how does the rate limit manner work for fixing 
> a FT split? how does each node get the rate limit? Will every node add 
> temporary flooding on a given number of links independently? If so, 
> there are lots of links to be added into the FT temporarily for fixing 
> the FT split. This may cause some issues.
>     In another thread "[Lsr] Min Links for Multiple Failures on 
> Flooding Topology", there is a solution for fixing the FT split using 
> almost minimum number of links.
> 
> Best Regards,
> Huaimo
> Regarding Issue #2 below, we had a healthy thread on this since Prague 
> and I believe have consensus that we WILL support LANs in the encoding 
> of the flooding topology (centralized mode). Authors need to agree on 
> changes to the draft which we will take offline and then publish an update.
> 
> Let me propose that we add something to sections 6.7.5, 6.7.9, and 
> 6.7.11
> like:
> 
> Addition of temporary flooding should be done with caution, as the 
> addition of excessive connectivity to the flooding topology may 
> trigger unwanted behavior. Routers SHOULD add temporary flooding in a 
> rate limited manner, if not configured otherwise.
> 
> <end snip>
> 
> (See attached email)
> 
> So I think we can consider these matters resolved - pending WG 
> feedback on the updated draft whenever it becomes available.
> 
>    Les
> 
> 
> > towards including the LANs in the signaling and rate limiting link 
> > addition
> during repair.
> > >
> > > Dissent? Discussion?
> > >
> > > Tony
> > >
> > >
> > link across the partition.  If that does not repair the partition in 
> > a while (LSP propagation time + SPF time), then add another link.
> > with a P2P link, the Area Leader cannot currently distinguish 
> > between the two links. This can be of importance if there are other
> > >>   systems also on the LAN that should be using their LAN 
> > >> interface for
> > flooding.
> > >>
> > as well as the system ID.  It would also make sense to rename the 
> > TLV to be the “IS-IS Area Node IDs TLV”.
> > >>
> > includes a pseudonode in the flooding topology, then all systems 
> > with an adjacency on that LAN should use the LAN as part of the

---

From: David Allan I <[email-masked]>
Date: Wed, 10 Apr 2019 13:30:34 +0000

Hi Tony:

So for my edification/education, this is a general behavior that in the absence of any specific algorithm is postulated.  (?)

The piece I do not quite get is "adding until fixed".  Is the working assumption that things have broken down to the point that there is no synchronization of the topology and the repairs are "blind"? Hence "adding until fixed" is adding until IGP exchange appears to have  minimized the changes in the LSDB from some previous state of the FT or some other criteria?  

Just checking
Dave

-----Original Message-----
From: Lsr <[email-masked]> On Behalf Of [email-masked]
Sent: Tuesday, April 9, 2019 12:13 PM
To: Huaimo Chen <[email-masked]>
Cc: Les Ginsberg <[email-masked]>; [email-masked]
Subject: Re: [Lsr] Fwd: Open issues with Dynamic Flooding

Hi Huaimo,

Tony

---

From: [email-masked]
Date: Wed, 10 Apr 2019 07:09:48 -0700

Hi Dave,

> So for my edification/education, this is a general behavior that in the absence of any specific algorithm is postulated.  (?)

Yes.

> The piece I do not quite get is "adding until fixed".  Is the working assumption that things have broken down to the point that there is no synchronization of the topology and the repairs are "blind”?

Yes.  We are using a centralized algorithm and have had multiple near simultaneous failures that have partitioned the flooding topology. While the area leader can (and probably will) compute a new flooding topology that repairs everything, it cannot be easily distributed across the partition of the flooding topology.

> Hence "adding until fixed" is adding until IGP exchange appears to have  minimized the changes in the LSDB from some previous state of the FT or some other criteria?  

Adding links temporarily to the FT should restore flooding, at which point an updated FT from the area leader can easily propagate.

Note that this is a convergence time issue: the new FT could slowly propagate, but it does require that each node at the partition unpack the new FT, install it, and then perform synchronization on the new links before proceeding.

Tony