Subject: [Lsr] Flooding across a network

From: <[email-masked]>
Date: Tue, 5 May 2020 15:27:56 +0000

Les,

> From: Lsr [mailto:[email-masked]] On Behalf Of Les Ginsberg (ginsberg)
> Sent: Monday, May 4, 2020 4:39 PM
[...]
> when only some nodes in the network support faster flooding the behavior of the whole network may not be "better" when faster flooding is enabled because it prolongs the period of LSDB inconsistency.  

1) Do you have data on this?

2) If not, can you provide an example where increasing the flooding rate on one adjacency prolongs the period of LSDB inconsistency across the network?

3) In the meantime, let's try the theoretical analysis on a simple scenario where a single LSP needs to be flooded across the network.

- Let's call Dij the time needed to flood the LSP from node i to the adjacent node j. Clearly Dij>0.
- Let's call k the node originating this LSP at t0=0s

>From t0, the LSDB is inconsistent across the network as all nodes but k are missing the LSP and hence only know about the 'old' topology.

Let's call  SPT(k) the SPT rooted on k, using Dij as the metric between adjacent nodes i and j. Let's call SP(k,i) the shortest path from k to i; and D(k,i) the shortest distance between k and i.

It seems that the time needed:
- for node j to learn about the LSP, and get in sync with k, is D(k,j)
- for all nodes across the network to learn about the LSP, and get in sync with k, is Max[for all j] D(k,j)

Then how can reducing the flooding delay on one adjacency could prolongs the period of LSDB inconsistency?
It seems to me that it can only improve/decrease it. Otherwise, this would mean that decreasing the cost on a link can increase the cost of the shortest path.

Note: I agree that there are other cases, such as  multiple LSPs originated by the same node, and multiple LSPs originated by multiple nodes, but let's start with the simple case. 

Thanks,
--Bruno

> -----Original Message-----
> From: Lsr [mailto:[email-masked]] On Behalf Of Les Ginsberg (ginsberg)
> Sent: Monday, May 4, 2020 4:39 PM
> 
> Henk -
> 
> Thanx for your thoughtful posts.
> I have read your later posts on this thread as well - but decided to reply to this one.
> Top posting for better readability.
> 
> There is broad agreement that faster flooding is desirable.
> There are now two proposals as to how to address the issue - neither of which is proposing to use TCP (or equivalent).
> 
> I have commented on why IS-IS flooding requirements are significantly different than that for which TCP is used.
> I think it is also useful to note that even the simple test case which Bruno reported on in last week's interim meeting demonstrated that without any changes to the protocol at all IS-IS was able to flood an order of magnitude faster than it commonly does today.
> This gives me hope that we are looking at the problem correctly and will not need "TCP".
> 
> Introducing a TCP based solution requires:
> 
> a)A major change to the adjacency formation logic
> 
> b)Removal of the independence of the IS-IS protocol from the address families whose reachability advertisements it supports - something which I think is a great strength of the protocol - particularly in environments where multiple address family support is needed
> 
> I really don't want to do either of the above.
> 
> Your comments regarding PSNP response times are quite correct - and both of the draft proposals discuss this - though I agree more detail will be required.
> It is intuitive that if you want to flood faster you also need to ACK faster - and probably even retransmit faster when that is needed.
> The basic relationship between retransmit interval and PSNP interval is expressed in ISO 10589:
> 
> " partialSNPInterval - This is the amount of time between periodic
        > action for transmission of Partial Sequence Number PDUs.
        > It shall be less than minimumLSPTransmission-Interval."
> 
> Of course ISO 10589 recommended values (2 seconds and 5 seconds respectively) associated with a much slower flooding rate and implementations I am aware of use values in this order of magnitude. These numbers need to be reduced if we are to flood faster, but the relationship between the two needs to remain the same.
> 
> It is also true - as you state - that sending ACKs more quickly will result in additional PDUs which need to be received/processed by IS-IS - and this has some impact. But I think it is reasonable to expect that an implementation which can support sending and receiving LSPs at a faster rate should also be able to send/receive PSNPs at a faster rate. But we still need to be smarter than sending one PSNP/one LSP in cases where we have a burst.
> 
> LANs are a more difficult problem than P2P - and thus far draft-ginsberg-lsr-isis-flooding-scale has been silent on this - but not because we aren't aware of this - just have focused on the P2P behavior first.
> What the best behavior on a LAN may be is something I am still considering. Slowing flooding down to the speed at which the slowest IS on the LAN can support may not be the best strategy - as it also slows down the propagation rate for systems downstream from the nodes on the LAN which can handle faster flooding - thereby having an impact on flooding speed throughout the network in a way which may be out of proportion. This is a smaller example of the larger issue that when only some nodes in the network support faster flooding the behavior of the whole network may not be "better" when faster flooding is enabled because it prolongs the period of LSDB inconsistency. More work needs to be done here...
> 
> In summary, I don't expect to have to "reinvent TCP" - but I do think you have provided a useful perspective for us to consider as we progress on this topic,
> 
> Thanx.
> 
    > Les
> 
> 
> > -----Original Message-----
> > From: Lsr <[email-masked]> On Behalf Of Henk Smit
> > Sent: Thursday, April 30, 2020 6:58 AM
> > To: [email-masked]
> > Subject: [Lsr] Why only a congestion-avoidance algorithm on the sender isn't
> > enough
> > 
> > 
> > Hello all,
> > 
> > Two years ago, Gunter Van de Velde and myself published this draft:
> > https://tools.ietf.org/html/draft-hsmit-lsr-isis-flooding-over-tcp-00
> > That started this discussion about flow/congestion control and ISIS
> > flooding.
> > 
> > My thoughts were that once we start implementing new algorithms to
> > optimize ISIS flooding speed, we'll end up with our own version of TCP.
> > I think most people here have a good general understanding of TCP.
> > But if not, this is a good overview how TCP does it:
> > https://en.wikipedia.org/wiki/TCP_congestion_control
> > 
> > 
> > What does TCP do:
> > ====
> > TCP does 2 things: flow control and congestion control.
> > 
> > 1) Flow control is: the receiver trying to prevent itself from being
> > overloaded. The receiver indicates, through the receiver-window-size
> > in the TCP acks, how much data it can or wants to receive.
> > 2) Congestion control is: the sender trying to prevent the links between
> > sender and receiver from being overloaded. The sender makes an educated
> > guess at what speed it can send.
> > 
> > 
> > The part we seem to be missing:
> > ====
> > For the sender to make a guess at what speed it can send, it looks at
> > how the transmission is behaving. Are there drops ? What is the RTT ?
> > Do drop-percentage and RTT change ? Do acks come in at the same rate
> > as the sender sends segments ? Are there duplicate acks ? To be able
> > to do this, the sender must know what to expect. How acks behave.
> > 
> > If you want an ISIS sender to make a guess at what speed it can send,
> > without changing the protocol, the only thing the sender can do is look
> > at the PSNPs that come back from the receiver. But the RTT of PSNPs can
> > not be predicted. Because a good ISIS implementation does not
> > immediately
> > send a PSNP when it receives a LSP. 1) the receiver should jitter the
> > PSNP,
> > like it should jitter all packets. And 2) the receiver should wait a
> > little
> > to see if it can combine multiple acks into a single PSNP packet.
> > 
> > In TCP, if a single segment gets lost, each new segment will cause the
> > receiver to send an ack with the seqnr of the last received byte. This
> > is called "duplicate acks". This triggers the sender to do
> > fast-retransmission. In ISIS, this can't be be done. The information
> > a sender can get from looking at incoming PSNPs is a lot less than what
> > TCP can learn from incoming acks.
> > 
> > 
> > The problem with sender-side congestion control:
> > ====
> > In ISIS, all we know is that the default retransmit-interval is 5
> > seconds.
> > And I think most implementations use that as the default. This means
> > that
> > the receiver of an LSP has one requirement: send a PSNP within 5
> > seconds.
> > For the rest, implementations are free to send PSNPs however and
> > whenever
> > they want. This means a sender can not really make conclusions about
> > flooding speed, dropped LSPs, capacity of the receiver, etc.
> > There is no ordering when flooding LSPs, or sending PSNPs. This makes
> > a sender-side algorithm for ISIS a lot harder.
> > 
> > When you think about it, you realize that a sender should wait the
> > full 5 seconds before it can make any real conclusions about dropped
> > LSPs.
> > If a sender looks at PSNPs to determine its flooding speed, it will
> > probably
> > not be able to react without a delay of a few seconds. A sender might
> > send
> > hunderds or thousands of LSPs in those 5 seconds, which might all or
> > partially be dropped, complicating matters even further.
> > 
> > 
> > A sender-sider algorithm should specify how to do PSNPs.
> > ====
> > So imho a sender-side only algorithm can't work just like that in a
> > multi-vendor environment. We must not only specify a congestion-control
> > algorithm for the sender. We must also specify for the receiver a more
> > specific algorithm how and when to send PSNPs. At least how to do PSNPs
> > under load.
> > 
> > Note that this might result in the receiver sending more (and smaller)
> > PSNPs.
> > More packets might mean more congestion (inside routers).
> > 
> > 
> > Will receiver-side flow-control work ?
> > ====
> > I don't know if that's enough. It will certainly help.
> > 
> > I think to tackle this problem, we need 3 parts:
> > 1) sender-side congestion-control algorithm
> > 2) more detailed algorithm on receiver when and how to send PSNPs
> > 3) receiver-side flow-control mechanism
> > 
> > As discussed at length, I don't know if the ISIS process on the
> > receiving
> > router can actually know if its running out of resources (buffers on
> > interfaces, linecards, etc). That's implementation dependent. A receiver
> > can definitely advertise a fixed value. So the sender has an upper bound
> > to use when doing congestion-control. Just like TCP has both a
> > flow-control
> > window and a congestion-control window, and a sender uses both. Maybe
> > the
> > receiver can even advertise a dynamic value. Maybe now, maybe only in
> > the
> > future. An advertised upper limit seems useful to me today.
> > 
> > 
> > What I didn't like about our own proposal (flooding over TCP):
> > ====
> > The problem I saw with flooding over TCP concerns multi-point networks
> > (LANs).
> > 
> > When flooding over a multi-point network, setting up TCP connections
> > introduces serious challenges. Who are the endpoints of the TCP
> > connections ?
> > Full mesh ? Or do all ISes on a LAN create a TCP-connection to the DIS ?
> > There is no backup DIS in ISIS (unlike OSPF). Things get messy quickly.
> > 
> > However, the other two proposals do not solve this problem either.
> > How will a sender-side congestion-avoidence algorithm determine whether
> > there were drops ? There are no acks (PSNPs) on a LAN. We assume most
> > LSPs
> > that are broadcasted are received by all other ISes on the LAN. There
> > are
> > no acks. Only after the DIS has sent its periodic CSNPs, ISes can send
> > PSNPs to request retransmissions. It seems impossible (or very hard) to
> > me for all ISes on a LAN to keep track of dropped LSPs and adjust their
> > sending speed accordingly.
> > 
> > When flooding on a LAN, the receiver-side algorithm seems best. Because
> > all ISes can see what the lowest advertised sending-speed is. And make
> > sure they send slow enough to not overload the slowest IS. I'm not sure
> > this is a good solution, but is seems easier and more realistic than
> > ISIS-flooding-over-TCP or sender-side congestion-avoidance.
> > 
> > 
> > My conclusion:
> > ====
> > Sender-side congestion-control won't work without specifying in more
> > detail how and when to send PSNPs.
> > Receiver-side flow-control will certainly help. I dont' know if it's
> > good enough. I don't know if advertising a static value is good enough.
> > But it's a start.
> > 
> > I still think we'll end up re-implementing a new (and weaker) TCP.
> > 
> > 
> > henk.
> > 
> > _______________________________________________
> > Lsr mailing list
> > [email-masked]
> > https://www.ietf.org/mailman/listinfo/lsr
> 
> _______________________________________________
> Lsr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lsr

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Tue, 5 May 2020 23:35:16 +0000

Bruno -

Seems like it was not too long ago that we were discussing this in person.  Ahhh...the good old days...

First, let's agree that the interesting case does not involve 1 or even a small number of LSPs. For those cases flooding speed does not matter.

The interesting cases involve a large number of LSPs (hundreds or thousands). And in such cases LFA/microloop avoidance techniques are not applicable.

Take the following simple topology:

   |  | ... |            |

     +---+             +---+

     | C |             | E |

     +---+             +---+

       |                 | 1000

     +---+             +---+

     | B |-------------| D |

     +---+   1000      +---+

       |                 |

       |                 |

        \               /

         \            /

          \         /

           \      /

             +---+

             | A |

             +---+

There is a topology northbound of C and E (not shown) and a topology southbound of A (not shown).

Cost on all links is 10 except B-D and D-E where cost is high.

C is a node with 1000 neighbors.

When all links are up, shortest path for all northbound destinations is via C.

All nodes in the network support fast flooding except for Node D.

Let's say fast flooding is 500 LSPs/second and slow flooding (Node D) is 20 LSPs/seconds.

If  Node C fails we have 1000 LSPs to flood.

All nodes except for D can receive these in 2 seconds (plus internode delay time).

D can receive LSPs in 50 seconds.

When A and B and all southbound nodes receive/process the LSP updates they will start sending traffic to Northbound destinations via D.

But for the better part of 50 seconds, Node D has yet to receive all LSP updates and still believes that shortest path is via B-C. It will loop traffic.

Had all nodes used slow flooding, it still would have taken 50 seconds to converge, but there would be significantly less looping. There could be a good amount of blackholing, but this is preferable to looping.

One can always come up with examples - based on a specific topology and a specific failure - where things might be better/worse/unchanged in the face of inconsistent flooding speed support.

But I hope this simple example illustrates the pitfalls.

    Les

> From: [email-masked] <[email-masked]>

> Sent: Tuesday, May 05, 2020 8:28 AM

> To: Les Ginsberg (ginsberg) <[email-masked]>; [email-masked]

> Subject: Flooding across a network

> > From: Lsr [mailto:[email-masked]] On Behalf Of Les Ginsberg

> (ginsberg)

> > when only some nodes in the network support faster flooding the behavior

> of the whole network may not be "better" when faster flooding is enabled

> because it prolongs the period of LSDB inconsistency.

> 2) If not, can you provide an example where increasing the flooding rate on

> one adjacency prolongs the period of LSDB inconsistency across the

> network?

> 3) In the meantime, let's try the theoretical analysis on a simple scenario

> where a single LSP needs to be flooded across the network.

> - Let's call Dij the time needed to flood the LSP from node i to the adjacent

> node j. Clearly Dij>0.

> >From t0, the LSDB is inconsistent across the network as all nodes but k are

> missing the LSP and hence only know about the 'old' topology.

> Let's call  SPT(k) the SPT rooted on k, using Dij as the metric between

> adjacent nodes i and j. Let's call SP(k,i) the shortest path from k to i; and

> D(k,i) the shortest distance between k and i.

> - for all nodes across the network to learn about the LSP, and get in sync with

> k, is Max[for all j] D(k,j)

> Then how can reducing the flooding delay on one adjacency could prolongs

> the period of LSDB inconsistency?

> It seems to me that it can only improve/decrease it. Otherwise, this would

> mean that decreasing the cost on a link can increase the cost of the shortest

> path.

> Note: I agree that there are other cases, such as  multiple LSPs originated by

> the same node, and multiple LSPs originated by multiple nodes, but let's start

> with the simple case.

> > From: Lsr [mailto:[email-masked]] On Behalf Of Les Ginsberg

> (ginsberg)

> > I have read your later posts on this thread as well - but decided to reply to

> this one.

> > There are now two proposals as to how to address the issue - neither of

> which is proposing to use TCP (or equivalent).

> > I have commented on why IS-IS flooding requirements are significantly

> different than that for which TCP is used.

> > I think it is also useful to note that even the simple test case which Bruno

> reported on in last week's interim meeting demonstrated that without any

> changes to the protocol at all IS-IS was able to flood an order of magnitude

> faster than it commonly does today.

> > This gives me hope that we are looking at the problem correctly and will not

> need "TCP".

> > b)Removal of the independence of the IS-IS protocol from the address

> families whose reachability advertisements it supports - something which I

> think is a great strength of the protocol - particularly in environments where

> multiple address family support is needed

> > Your comments regarding PSNP response times are quite correct - and

> both of the draft proposals discuss this - though I agree more detail will be

> required.

> > It is intuitive that if you want to flood faster you also need to ACK faster -

> and probably even retransmit faster when that is needed.

> > The basic relationship between retransmit interval and PSNP interval is

> expressed in ISO 10589:

> > Of course ISO 10589 recommended values (2 seconds and 5 seconds

> respectively) associated with a much slower flooding rate and

> implementations I am aware of use values in this order of magnitude. These

> numbers need to be reduced if we are to flood faster, but the relationship

> between the two needs to remain the same.

> > It is also true - as you state - that sending ACKs more quickly will result in

> additional PDUs which need to be received/processed by IS-IS - and this has

> some impact. But I think it is reasonable to expect that an implementation

> which can support sending and receiving LSPs at a faster rate should also be

> able to send/receive PSNPs at a faster rate. But we still need to be smarter

> than sending one PSNP/one LSP in cases where we have a burst.

> > LANs are a more difficult problem than P2P - and thus far draft-ginsberg-lsr-

> isis-flooding-scale has been silent on this - but not because we aren't aware

> of this - just have focused on the P2P behavior first.

> > What the best behavior on a LAN may be is something I am still considering.

> Slowing flooding down to the speed at which the slowest IS on the LAN can

> support may not be the best strategy - as it also slows down the propagation

> rate for systems downstream from the nodes on the LAN which can handle

> faster flooding - thereby having an impact on flooding speed throughout the

> network in a way which may be out of proportion. This is a smaller example

> of the larger issue that when only some nodes in the network support faster

> flooding the behavior of the whole network may not be "better" when faster

> flooding is enabled because it prolongs the period of LSDB inconsistency.

> More work needs to be done here...

> > In summary, I don't expect to have to "reinvent TCP" - but I do think you

> have provided a useful perspective for us to consider as we progress on this

> topic,

> > > From: Lsr <[email-masked]<mailto:[email-masked]>> On Behalf Of Henk Smit

> > > To: [email-masked]<mailto:[email-masked]>

> > > Subject: [Lsr] Why only a congestion-avoidance algorithm on the sender

> isn't

> > > sender and receiver from being overloaded. The sender makes an

> educated

> > > multi-vendor environment. We must not only specify a congestion-

> control

> > > specific algorithm how and when to send PSNPs. At least how to do

> PSNPs

> > > window and a congestion-control window, and a sender uses both.

> Maybe

> > > How will a sender-side congestion-avoidence algorithm determine

> whether

> > > When flooding on a LAN, the receiver-side algorithm seems best.

> Because

> > > [email-masked]<mailto:[email-masked]>

> > [email-masked]<mailto:[email-masked]>

> __________________________________________________________

> __________________________________________________________

> _____

> Ce message et ses pieces jointes peuvent contenir des informations

> confidentielles ou privilegiees et ne doivent donc

> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce

> message par erreur, veuillez le signaler

> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages

> electroniques etant susceptibles d'alteration,

> Orange decline toute responsabilite si ce message a ete altere, deforme ou

> falsifie. Merci.

> This message and its attachments may contain confidential or privileged

> information that may be protected by law;

> If you have received this email in error, please notify the sender and delete

> this message and its attachments.

> As emails may be altered, Orange is not liable for messages that have been

> modified, changed or falsified.

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 6 May 2020 02:03:58 +0200

Hi Les,

A side comment but your example shows another - one may say even much more
serious issue.

Assume we have LFA/TI-LFA enabled in the network and precomputed on B which
get's activated and shifts traffic to E when detects that C is down.
Detection is fast .. 10s-100s of milliseconds.

Now if B converges fast and recomputes topology much faster then D it may
remove protection and send packets to D natively. Well clearly as we
established D is slow and will loop it back.

That is why I mentioned the other day that a fast control plane is not
always a good thing (I am sure many will say the opposite - but it is ok
;).

But this proves that consistent convergence time in a domain is rather a
good thing regardless if it takes 2 sec or 50 sec on all nodes.

Best,
Robert.

On Wed, May 6, 2020 at 1:35 AM Les Ginsberg (ginsberg) <ginsberg=
[email-masked]> wrote:

> First, let's agree that the interesting case does not involve 1 or even a
> small number of LSPs. For those cases flooding speed does not matter.
>
> The interesting cases involve a large number of LSPs (hundreds or
> thousands). And in such cases LFA/microloop avoidance techniques are not
> applicable.
>
>
>
> *   |  | ... |            |*
> *     +---+             +---+*
> *     | C |             | E |*
> *     +---+             +---+*
> *       |                 | 1000*
> *     +---+             +---+*
> *     | B |-------------| D |*
>
> *     +---+   1000      +---+*
>
> *       |                 |*
>
> *       |                 |*
>
> *        \               /*
>
> *         \            /*
>
> *          \         /*
>
> *           \      /*
>
> *             +---+*
>
> *             | A |*
>
> *             +---+*
>
>
>
> Let’s say fast flooding is 500 LSPs/second and slow flooding (Node D) is
> 20 LSPs/seconds.
>
> When A and B and all southbound nodes receive/process the LSP updates they
> will start sending traffic to Northbound destinations via D.
>
> But for the better part of 50 seconds, Node D has yet to receive all LSP
> updates and still believes that shortest path is via B-C. It will loop
> traffic.
>
>
>
> Had all nodes used slow flooding, it still would have taken 50 seconds to
> converge, but there would be significantly less looping. There could be a
> good amount of blackholing, but this is preferable to looping.
>
>
>
> One can always come up with examples – based on a specific topology and a
> specific failure - where things might be better/worse/unchanged in the face
> of inconsistent flooding speed support.
>
> > > From: Lsr [mailto:[email-masked] <[email-masked]>] On
> Behalf Of Les Ginsberg
>
> > (ginsberg)
>
> > D(k,i) the shortest distance between k and i..
>
> >
>
> > > From: Lsr [mailto:[email-masked] <[email-masked]>] On
> Behalf Of Les Ginsberg
>
> > (ginsberg)
>
> > > > send a PSNP when it receives a LSP.. 1) the receiver should jitter
> the
>
> > > > PSNP,
>

---

From: Christian Hopps <[email-masked]>
Date: Tue, 5 May 2020 23:48:22 -0400

[as WG member]

I think it would be more productive if we stay focused on trying to improve flooding speed/efficiency here. How about let's get some of the proposals being mulled over actually written, and provide some data, and leave all the hand-wringing and theorizing about being too-successful for after we've shown we could be? :)

Thanks,
Chris.

> On May 5, 2020, at 8:03 PM, Robert Raszuk <[email-masked]> wrote:
> 
> But this proves that consistent convergence time in a domain is rather a good thing regardless if it takes 2 sec or 50 sec on all nodes. 
>

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 6 May 2020 09:52:08 +0200

Christian,

It is not about "hand-wringing and theorizing about being too-successful".

It is about impact to the overall network service when you make ISIS
convergence not consistent across nodes of the topology, I think our goal
is not to converge ISIS faster by improving flooding speed ... it is much
more about how to deliver user packets with minimal disruption upon
topology changes.

Best,
R.

On Wed, May 6, 2020 at 5:48 AM Christian Hopps <[email-masked]> wrote:

> [as WG member]
>
> I think it would be more productive if we stay focused on trying to
> improve flooding speed/efficiency here. How about let's get some of the
> proposals being mulled over actually written, and provide some data, and
> leave all the hand-wringing and theorizing about being too-successful for
> after we've shown we could be? :)
>
> Thanks,
> Chris.
>
>

---

From: <[email-masked]>
Date: Wed, 6 May 2020 13:20:51 +0000

Les,

From: Les Ginsberg (ginsberg) [mailto:[email-masked]]
Sent: Wednesday, May 6, 2020 1:35 AM
To: DECRAENE Bruno TGI/OLN; [email-masked]
Subject: RE: Flooding across a network

Bruno -

[Bruno] Indeed, may be not to the point of concluding. Indeed.

   |  | ... |            |

     +---+             +---+

     | C |             | E |

     +---+             +---+

       |                 | 1000

     +---+             +---+

     | B |-------------| D |

     +---+   1000      +---+

       |                 |

       |                 |

        \               /

         \            /

          \         /

           \      /

             +---+

             | A |

             +---+

[Bruno] Thanks for your example. Agreed so far.

[Bruno] May I remind you that we are discussing IS-IS flooding in order to sync LSDB (LSP database). That is already a big enough subject. It does not including FIB (updates), nor IP forwarding.

Quoting you "when only some nodes in the network support faster flooding the behavior of the whole network may not be "better" when faster flooding is enabled because it prolongs the period of LSDB inconsistency."

Taking your own examples, in both cases (all nodes support fast flooding; all nodes but D support fast flooding) the period of LSDB inconsistency is 50 seconds. Hence this example does not illustrate your statement.

[Bruno] You are using an example where ordering FIB updates across the network, e.g. as per [1], allows to reduce _FIB_ inconsistency across the path/network. And you seem to conclude from this that this translates to LSDB update ordering. Those are two different things. In this thread, I'd suggest that we focus on IGP flooding and LSDB sync only. (*)

[1] https://tools.ietf.org/html/rfc6976

(*) We can discuss loop free IGP converge in a different thread if you want. IMO, the use of segment routing/source routing is better than oFIB. But at some point, it still relies on fast flooding when multiple LSPs are involved. (and I mean _fast_ not _ordered_)

--Bruno

    Les

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Wed, 6 May 2020 14:14:22 +0000

Bruno -

I am somewhat at a loss to understand your comments.
The example is straightforward and does not need to consider FIB update time nor the ordering of prefix updates on different nodes.

Consider the state of Node B and Node D at various time points from the trigger event.

T+ 2 seconds:
-----------------
B has received all LSP Updates. It triggers an SPF and for all Northbound destinations previously reachable via C it installs paths via D.
Let's assume it take 5 seconds to update the forwarding plane.

D has received 40 of the 1000 LSP updates. It triggers an SPF and finds that all Northbound destinations are reachable via B-C. It makes no changes to the forwarding plane.

T+7 seconds
-----------------
B has completed FIB updates. Traffic to all Northbound destinations is being forwarded via D.

D has now received 140 of the 1000 LSP updates. Entries in its forwarding plane for Northbound destinations still point to B.

We have a loop.

T + 30 seconds
--------------------
D has now received 600 of the 1000 LSP updates. Still no changes to its forwarding plane.
Traffic to Northbound destinations is still looping.

T+ 50 seconds
-------------------
D has finally received all 1000 LSP updates.
It triggers (another) SPF and calculates paths to Northbound destinations via E. It begins to update its forwarding plane.
Let's assume this will take 5 seconds.

T + 55 seconds
--------------------
D has completed forwarding plane updates - no more looping.

That is all I am trying to illustrate.

If you want to start arguing that node protecting LFAs + microloop avoidance could help (NOTE I explicitly  took those out of the example for simplicity) - it is easy enough to change the example to include multiple node failures or a node failure plus some northbound link failures on other nodes.
The point here is to look at the impact of long-lived LSDB inconsistency which results when some nodes support flooding an order of magnitude faster flooding than other nodes - which is what you asked me to clarify.

   Les

From: [email-masked] <[email-masked]>
Sent: Wednesday, May 06, 2020 6:21 AM
To: Les Ginsberg (ginsberg) <[email-masked]>
Cc: [email-masked]
Subject: RE: Flooding across a network

Les,

From: Les Ginsberg (ginsberg) [mailto:[email-masked]]
Sent: Wednesday, May 6, 2020 1:35 AM
To: DECRAENE Bruno TGI/OLN; [email-masked]<mailto:[email-masked]>
Subject: RE: Flooding across a network

Bruno -

   |  | ... |            |

     +---+             +---+

     | C |             | E |

     +---+             +---+

       |                 | 1000

     +---+             +---+

     | B |-------------| D |

     +---+   1000      +---+

       |                 |

       |                 |

        \               /

         \            /

          \         /

           \      /

             +---+

             | A |

             +---+

--Bruno

    Les

> From: [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>

> To: Les Ginsberg (ginsberg) <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>

Thank you.

---

From: <[email-masked]>
Date: Wed, 6 May 2020 14:33:22 +0000

Les,

From: Les Ginsberg (ginsberg) [mailto:[email-masked]]
Sent: Wednesday, May 6, 2020 4:14 PM
To: DECRAENE Bruno TGI/OLN
Cc: [email-masked]
Subject: RE: Flooding across a network

Bruno -

I am somewhat at a loss to understand your comments.
The example is straightforward and does not need to consider FIB update time nor the ordering of prefix updates on different nodes.
[Bruno] The example is straightforward but you are referring to FIB and IP packets forwarding as per those FIBs.
I'd like we focus on LSP flooding and LSDB consistency.

T+ 2 seconds:
-----------------

T+7 seconds
-----------------

We have a loop.

T + 30 seconds
--------------------

T+ 50 seconds
-------------------

T + 55 seconds
--------------------

If you want to start arguing that node protecting LFAs + microloop avoidance could help (NOTE I explicitly  took those out of the example for simplicity) - it is easy enough to change the example to include multiple node failures or a node failure plus some northbound link failures on other nodes.
[Bruno] I'm not talking about LFA/FRR. And with regards to microloops avoidance, some algorithms can handle any graph transition so including multiple node failures.

But again, let's stick to LSP flooding and LSDB consistency. (you are the one speaking about microloops in the forwarding plane).

The point here is to look at the impact of long-lived LSDB inconsistency which results when some nodes support flooding an order of magnitude faster flooding than other nodes - which is what you asked me to clarify.
[Bruno] No. I asked you to clarify why having a node with faster flooding could prolongs the period of LSDB inconsistency.

Again, with you own words: "when only some nodes in the network support faster flooding the behavior of the whole network may not be "better" when faster flooding is enabled because it prolongs the period of LSDB inconsistency."
And with less words: "when only some nodes in the network support faster flooding [...]  it prolongs the period of LSDB inconsistency."

--Bruno

   Les

Les,

Bruno -

   |  | ... |            |

     +---+             +---+

     | C |             | E |

     +---+             +---+

       |                 | 1000

     +---+             +---+

     | B |-------------| D |

     +---+   1000      +---+

       |                 |

       |                 |

        \               /

         \            /

          \         /

           \      /

             +---+

             | A |

             +---+

--Bruno

    Les

Thank you.

---

From: Christian Hopps <[email-masked]>
Date: Wed, 6 May 2020 10:48:27 -0400

Bruno persistence has made me realize something fundamental here.

The minute the LSP originator changes the LSP and floods it you have LSDB inconsistency. That is going to last until the last node in the network has updated it's LSDB.

Les is pointing out that LSDB inconsistency can be bad in certain circumstances e.g., if a critical node is slow and thus inconsistent.

I believe the right way to fix this is a simple one, help the operator flag the broken router software/hardware for replacement, but otherwise IS-IS should just try to do the best job it can do to which is to flood around the problem (i.e., flood as optimally as possible).

Thanks,
Chris.
[as WG member]

> On May 6, 2020, at 10:33 AM, [email-masked] wrote:
> 
> Les,
>  
> Bruno –
> I’d like we focus on LSP flooding and LSDB consistency.
>  
> Let’s assume it take 5 seconds to update the forwarding plane.
>  
> D has finally received all 1000 LSP updates..
> Let’s assume this will take 5 seconds..
>  
> T + 55 seconds
> --------------------
> D has completed forwarding plane updates – no more looping.
>  
> If you want to start arguing that node protecting LFAs + microloop avoidance could help (NOTE I explicitly  took those out of the example for simplicity) – it is easy enough to change the example to include multiple node failures or a node failure plus some northbound link failures on other nodes.
> [Bruno] I’m not talking about LFA/FRR. And with regards to microloops avoidance, some algorithms can handle any graph transition so including multiple node failures.
>  
> But again, let’s stick to LSP flooding and LSDB consistency. (you are the one speaking about microloops in the forwarding plane).
>  
> The point here is to look at the impact of long-lived LSDB inconsistency which results when some nodes support flooding an order of magnitude faster flooding than other nodes – which is what you asked me to clarify.
> Again, with you own words: “when only some nodes in the network support faster flooding the behavior of the whole network may not be "better" when faster flooding is enabled because it prolongs the period of LSDB inconsistency.”
> And with less words: “when only some nodes in the network support faster flooding […]  it prolongs the period of LSDB inconsistency.”
>  
> --Bruno
>  
>    Les
>  
>  
>  
> To: DECRAENE Bruno TGI/OLN; lsr@ietf..org
> Let’s say fast flooding is 500 LSPs/second and slow flooding (Node D) is 20 LSPs/seconds.
> Quoting you “when only some nodes in the network support faster flooding the behavior of the whole network may not be "better" when faster flooding is enabled because it prolongs the period of LSDB inconsistency.”
>  
> Hence I’m restating my questions:
>  
> [Bruno] You are using an example where ordering FIB updates across the network, e.g. as per [1], allows to reduce _FIB_ inconsistency across the path/network. And you seem to conclude from this that this translates to LSDB update ordering. Those are two different things. In this thread, I’d suggest that we focus on IGP flooding and LSDB sync only. (*)
> One can always come up with examples – based on a specific topology and a specific failure - where things might be better/worse/unchanged in the face of inconsistent flooding speed support.
> > > > sending speed accordingly..
> > > >

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Wed, 6 May 2020 14:51:58 +0000

Bruno -

The key in this example is when all nodes in the network realize that there are no longer any northbound paths via C.
I am willing to stipulate that nodes that can receive LSPs faster will also be able to flood LSPs faster. If that isn't the case we might as well all stop working on faster flooding.

In the example, all nodes other than D will have reconverged to a topology with no northbound paths via C in time (2 seconds + network propagation time + FIB update time).
How long would you like to allow for this? 10 seconds? 15 seconds?
That still leaves Node D in an inconsistent state for 35+ seconds where it thinks there are still Northbound paths via C.

If all nodes support the same flooding rate (faster or slower) they will all reach an LSDB state where there are no Northbound paths via C within a few seconds of each other.
So - yes - there can still be loops even with consistent flooding rates - but they will be an order of magnitude shorter (a few seconds vs 50 seconds in the example).

  Les

From: Lsr <[email-masked]> On Behalf Of [email-masked]
Sent: Wednesday, May 06, 2020 7:33 AM
To: Les Ginsberg (ginsberg) <[email-masked]>
Cc: [email-masked]
Subject: Re: [Lsr] Flooding across a network

Les,

Cc: [email-masked]<mailto:[email-masked]>

Bruno -

T+ 2 seconds:
-----------------

T+7 seconds
-----------------

We have a loop.

T + 30 seconds
--------------------

T+ 50 seconds
-------------------
D has finally received all 1000 LSP updates..
It triggers (another) SPF and calculates paths to Northbound destinations via E. It begins to update its forwarding plane.
Let's assume this will take 5 seconds..

T + 55 seconds
--------------------

--Bruno

   Les

From: [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>
Sent: Wednesday, May 06, 2020 6:21 AM
To: Les Ginsberg (ginsberg) <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: RE: Flooding across a network

Les,

From: Les Ginsberg (ginsberg) [mailto:[email-masked]]
Sent: Wednesday, May 6, 2020 1:35 AM
To: DECRAENE Bruno TGI/OLN; lsr@ietf..org<mailto:[email-masked]>
Subject: RE: Flooding across a network

Bruno -

   |  | ... |            |

     +---+             +---+

     | C |             | E |

     +---+             +---+

       |                 | 1000

     +---+             +---+

     | B |-------------| D |

     +---+   1000      +---+

       |                 |

       |                 |

        \               /

         \            /

          \         /

           \      /

             +---+

             | A |

             +---+

--Bruno

    Les

Thank you.

Thank you.

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Wed, 6 May 2020 14:58:38 +0000

Chris -

I agree.

https://tools.ietf.org/html/draft-ginsberg-lsr-isis-flooding-scale-02#section-3.1 states:

" Inability to handle LSP reception at the
   targeted flooding rate should be viewed as an error condition which
   should be reported.  If this condition persists, it indicates that
   the network is provisioned in a way which does not support optimal
   convergence.  Steps need to be taken to resolve this issue.  Such
   steps could include upgrading the routers that demonstrate this
   condition consistently, altering the configuration on the problematic
   routers or altering the position of the problematic routers in the
   network so as to reduce the overall load on those routers, or
   reducing the LSP transmission rate network-wide."

   Les

> From: Christian Hopps <[email-masked]>
> Sent: Wednesday, May 06, 2020 7:48 AM
> To: [email-masked]
> Cc: Christian Hopps <[email-masked]>; Les Ginsberg (ginsberg)
> <[email-masked]>; [email-masked]
> The minute the LSP originator changes the LSP and floods it you have LSDB
> inconsistency. That is going to last until the last node in the network has
> updated it's LSDB.
> 
> Les is pointing out that LSDB inconsistency can be bad in certain
> circumstances e.g., if a critical node is slow and thus inconsistent.
> 
> I believe the right way to fix this is a simple one, help the operator flag the
> broken router software/hardware for replacement, but otherwise IS-IS
> should just try to do the best job it can do to which is to flood around the
> problem (i.e., flood as optimally as possible).
> 
> Thanks,
> Chris.
> [as WG member]
> 
> 
> > The example is straightforward and does not need to consider FIB update
> time nor the ordering of prefix updates on different nodes.
> > B has received all LSP Updates. It triggers an SPF and for all Northbound
> destinations previously reachable via C it installs paths via D.
> > D has received 40 of the 1000 LSP updates. It triggers an SPF and finds that
> all Northbound destinations are reachable via B-C. It makes no changes to
> the forwarding plane.
> >
> > T+7 seconds
> > -----------------
> > D has now received 140 of the 1000 LSP updates. Entries in its forwarding
> plane for Northbound destinations still point to B.
> >
> > We have a loop.
> >
> > T + 30 seconds
> > --------------------
> > It triggers (another) SPF and calculates paths to Northbound destinations
> via E. It begins to update its forwarding plane.
> > If you want to start arguing that node protecting LFAs + microloop
> avoidance could help (NOTE I explicitly  took those out of the example for
> simplicity) – it is easy enough to change the example to include multiple node
> failures or a node failure plus some northbound link failures on other nodes.
> > [Bruno] I’m not talking about LFA/FRR. And with regards to microloops
> avoidance, some algorithms can handle any graph transition so including
> multiple node failures.
> >
> > But again, let’s stick to LSP flooding and LSDB consistency. (you are the one
> speaking about microloops in the forwarding plane).
> >
> > The point here is to look at the impact of long-lived LSDB inconsistency
> which results when some nodes support flooding an order of magnitude
> faster flooding than other nodes – which is what you asked me to clarify.
> > [Bruno] No. I asked you to clarify why having a node with faster flooding
> could prolongs the period of LSDB inconsistency.
> >
> > Again, with you own words: “when only some nodes in the network
> support faster flooding the behavior of the whole network may not be
> "better" when faster flooding is enabled because it prolongs the period of
> LSDB inconsistency.”
> > And with less words: “when only some nodes in the network support
> faster flooding […]  it prolongs the period of LSDB inconsistency.”
> >
> > --Bruno
> >
> >    Les
> >
> >
> >
> > First, let's agree that the interesting case does not involve 1 or even a small
> number of LSPs. For those cases flooding speed does not matter.
> > When A and B and all southbound nodes receive/process the LSP updates
> they will start sending traffic to Northbound destinations via D.
> updates and still believes that shortest path is via B-C. It will loop traffic.
> >
> > [Bruno] May I remind you that we are discussing IS-IS flooding in order to
> sync LSDB (LSP database). That is already a big enough subject. It does not
> including FIB (updates), nor IP forwarding.
> >
> > Quoting you “when only some nodes in the network support faster
> flooding is enabled because it prolongs the period of LSDB inconsistency.”
> >
> > Taking your own examples, in both cases (all nodes support fast flooding;
> all nodes but D support fast flooding) the period of LSDB inconsistency is 50
> seconds. Hence this example does not illustrate your statement.
> >
> > [Bruno] You are using an example where ordering FIB updates across the
> network, e.g. as per [1], allows to reduce _FIB_ inconsistency across the
> path/network. And you seem to conclude from this that this translates to
> LSDB update ordering. Those are two different things. In this thread, I’d
> suggest that we focus on IGP flooding and LSDB sync only. (*)
> > (*) We can discuss loop free IGP converge in a different thread if you want.
> IMO, the use of segment routing/source routing is better than oFIB. But at
> some point, it still relies on fast flooding when multiple LSPs are involved.
> (and I mean _fast_ not _ordered_)
> >
> > --Bruno
> >
> > One can always come up with examples – based on a specific topology and
> a specific failure - where things might be better/worse/unchanged in the
> face of inconsistent flooding speed support.

---

From: Christian Hopps <[email-masked]>
Date: Wed, 6 May 2020 12:50:41 -0400

I guess all this discussion has to do with this other part of that document:

   "o  Limitations on the processing rate of incoming control traffic

   However, intentionally using different flooding rates on different
   interfaces increases the possibility of longer periods of LSDB
   inconsistency which in turn delays network wide convergence."

[chopps: but using dynamic rates (thus different), allows one to flood around a bad node in the network, among other things, see below]

   Many implementations provide knobs to control the rate of LSP
   flooding on a per interface basis.  To the extent that this serves as
   a flow control mechanism, this may reduce the number of dropped LSPs
   during high activity bursts and thereby reduce the number of LSP
   retransmissions required.  As LSP retransimssion timers are typically
   long (multiple seconds), this may result in shorter convergence times
   than if the LSP burst was uncontrolled.  But if the performance
   characteristics of routers in the network are such that some routers
   consistently accept and process fewer LSPs/second than other routers,
   convergence will be degraded.  Tuning LSP transmission timers on a
   per interface basis will never provide optimal convergence.
   Consistent flooding rates should be used on all interfaces."

This last bit is what people are disagreeing with since we are now considering dynamically updating the flooding rate to be locally optimal, and I believe that last part is targeted at that concept.

The document is implying that it is not only minimizing LSDB inconsistency in the network by flooding at the same rate on all interfaces, but it is asserting that doing differently will never provide optimal convergence. I believe that assertion is false; it is ignoring topological differences. Consider this network:

+-+
| | --- [C0] ... [x0]
|A|                |
| | ------------ [C1]
+-+

If the same flooding rate is used everywhere, or even just on As interfaces, and C1 and x0 (prior to the LSDB change) used to reach [A] through [C1->A] but after the LSDB change will reach [A] through [x0->..C0] you will have guaranteed a loop between [C1] and [x0] to reach [A] for all the unnecessary flooding delay the document is adding through equality requirements. 

If instead you flood optimally (allowing for going as fast as possible) then the loop is minimized to the difference between [A]-[C1] flood time and [A]-[C0]-...-[x0] flood time. Clearly an optimal flooding rate (even if unequal) minimizes the loop between [C1] and [x0]. This is just one of any number of similar topologies that are possible (imagine a bunch of sub-topologies that replicate the issue above).

Anyway I'm with Tony Li here, we need some running code and data to look at with various scenarios.

Thanks,
Chris.
[as WG member]

> On May 6, 2020, at 10:58 AM, Les Ginsberg (ginsberg) <ginsberg=[email-masked]> wrote:
> 
> Chris -
> 
> I agree.
>

---

From: <[email-masked]>
Date: Wed, 6 May 2020 17:09:41 +0000

> From: Christian Hopps [mailto:[email-masked]] 
> 
> The minute the LSP originator changes the LSP and floods it you have LSDB inconsistency. 

Exactly my point. Thank you Chris.
I would even say: "The minute the LSP originator changes the LSP then you have LSDB inconsistency." But no big deal if there is disagreement on this detail.

> That is going to last until the last node in the network has updated it's LSDB.

Absolutely.
So the faster we flood, the shorter the LSBD inconsistency.

Now IMO, even if a single/few nodes flood faster, there is a chance of shortening the LSDB inconsistency. But in all cases, I don't see how this could make the LSDB inconsistency longer.

> I believe the right way to fix this is a simple one, help the operator flag the broken router software/hardware for replacement, but otherwise IS-IS should just try to do the best job it can do to…

+1
On a side note, I would not call a router flooding slowly as "broken". I find it understandable that in a given network there are different type of routers (core vs aggregation), different roles (P having 50 IGP adjacencies with 50 PEs vs PE having only 2 IGP adjacencies with 2 P), different hardware generations, different software, different vendors with different perspectives/markets.

Thank you Chris.

--Bruno

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Wed, 6 May 2020 17:53:15 +0000

Bruno -

I am sorry it has been so difficult for us to understand each other. I am trying my best.

Look at it this way:

You are the customer. 😊
I am the vendor. 

The failure scenario I describe below happens and you notice that all Northbound destinations loop for 35 seconds whenever fast flooding is enabled.
I think you are going to complain about this - to me. 😊

And I am going to tell you that this is a consequence of enabling fast flooding in the presence of a node which does not support it. Your options to reduce the period of looping will be:

1)Upgrade the slow node to support faster flooding
2)Disable fast flooding
3)Redesign your network

    Les

> Sent: Wednesday, May 06, 2020 10:10 AM
> To: Christian Hopps <[email-masked]>
> Cc: Les Ginsberg (ginsberg) <[email-masked]>; [email-masked]
> Subject: RE: [Lsr] Flooding across a network
> 
> inconsistency.
> I would even say: "The minute the LSP originator changes the LSP then you
> have LSDB inconsistency." But no big deal if there is disagreement on this
> detail.
> 
> Now IMO, even if a single/few nodes flood faster, there is a chance of
> shortening the LSDB inconsistency. But in all cases, I don't see how this could
> make the LSDB inconsistency longer.
> 
> 
> On a side note, I would not call a router flooding slowly as "broken". I find it
> understandable that in a given network there are different type of routers
> (core vs aggregation), different roles (P having 50 IGP adjacencies with 50 PEs
> vs PE having only 2 IGP adjacencies with 2 P), different hardware
> generations, different software, different vendors with different
> perspectives/markets.
> 
> Thank you Chris.
> 
> --Bruno
> >
> > Thanks,
> > Chris.
> > [as WG member]
> >
> >
> > > D has received 40 of the 1000 LSP updates. It triggers an SPF and finds
> that all Northbound destinations are reachable via B-C. It makes no changes
> to the forwarding plane.
> > >
> > > T+7 seconds
> > > -----------------
> > > But again, let’s stick to LSP flooding and LSDB consistency. (you are the
> one speaking about microloops in the forwarding plane).
> > >
> > > When A and B and all southbound nodes receive/process the LSP
> updates they will start sending traffic to Northbound destinations via D.
> > > (*) We can discuss loop free IGP converge in a different thread if you
> want. IMO, the use of segment routing/source routing is better than oFIB.
> But at some point, it still relies on fast flooding when multiple LSPs are
> involved. (and I mean _fast_ not _ordered_)
> > >
> > > --Bruno
> > >
> > > One can always come up with examples – based on a specific topology
> and a specific failure - where things might be better/worse/unchanged in the
> > > pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu
> ce message par erreur, veuillez le signaler
> > > pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu
> ce message par erreur, veuillez le signaler

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 6 May 2020 14:19:26 -0400

Les, maybe I am missing your point, but it sounds like what you are 
asking for is a (better?) version of the micro-loop prevention work, so 
as to mitigate the interaction between inconsistent convergence and 
fast-reroute?

Yours,
Joel

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Wed, 6 May 2020 18:25:19 +0000

Joel -

No - I am not "asking" for anything.

I am simply trying to demonstrate that a deployment that has nodes with significantly different flooding rate support can encounter long periods of looping in certain failure scenarios - and that a similar period of looping would NOT occur if all nodes supported a consistent flooding rate.
Do not read anything else into this.

   Les

> From: Joel M. Halpern <[email-masked]>
> Sent: Wednesday, May 06, 2020 11:19 AM
> > The failure scenario I describe below happens and you notice that all
> Northbound destinations loop for 35 seconds whenever fast flooding is
> enabled.
> > And I am going to tell you that this is a consequence of enabling fast
> flooding in the presence of a node which does not support it. Your options to
> reduce the period of looping will be:
> >
> >>>> [Bruno] The example is straightforward but you are referring to FIB and
> IP
> >> packets forwarding as per those FIBs.
> >>>> Consider the state of Node B and Node D at various time points from
> the
> >> trigger event.
> >>>>
> >>>> T+ 2 seconds:
> >>>> -----------------
> >>>> Seems like it was not too long ago that we were discussing this in
> person.
> >> Ahhh...the good old days...
> >>>> When all links are up, shortest path for all northbound destinations is
> via
> >> C.
> 20
> >> LSPs/seconds.
> >>>>> >From t0, the LSDB is inconsistent across the network as all nodes but
> k
> >> are
> >>>>> - for all nodes across the network to learn about the LSP, and get in
> sync
> >> with
> >>>>>> There are now two proposals as to how to address the issue -
> neither
> >> of
> >>>>> reported on in last week's interim meeting demonstrated that
> without
> >> any
> >>>>> both of the draft proposals discuss this - though I agree more detail
> will
> >> be
> >>>>> required.
> >>>>>> It is also true - as you state - that sending ACKs more quickly will
> result
> >> in
> >>>>> additional PDUs which need to be received/processed by IS-IS - and
> this
> >> has
> >>>>> Slowing flooding down to the speed at which the slowest IS on the
> LAN
> >> can
> >>>>> flooding the behavior of the whole network may not be "better"
> when
> >> faster
> >>>>> have provided a useful perspective for us to consider as we progress
> on
> >> this
> >>>>> topic,
> >>>>>>
> >>>>>> Thanx.
> >>>>>>
> >>>>>      > Les
> >>>>>>
> >>>>>>
> >>>>>>> My thoughts were that once we start implementing new
> algorithms
> >> to
> >>>>>>> how the transmission is behaving. Are there drops ? What is the
> RTT
> >> ?
> >>>>>>> there were drops ? There are no acks (PSNPs) on a LAN. We
> assume
> >> most
> >>>>>>> LSPs
> >>>> Orange decline toute responsabilite si ce message a ete altere,
> deforme
> >> ou falsifie. Merci.
> >>>>
> >>>> Orange decline toute responsabilite si ce message a ete altere,
> deforme
> >> ou falsifie. Merci.
> >>>>

---

From: Tony Przygienda <[email-masked]>
Date: Wed, 6 May 2020 11:31:44 -0700

This is not a simple let's-build-a-better-mechanism problem, this is an
epistemology problem and uneven information diffusion cannot be fixed by
magic when dealing with total distributed computation. Traditional
Link-state basically only works because we assume an epsilon consistently
(correct term for "eventual" consistency) with very small epsilon across
the whole topology. It used that epsilon in seconds was good, now epsilon
in hundreds of milliseconds is perceived problematic. If we start to build
things where the gradient varies across the network we will have unintended
consequences of distributed computation Les describes that cannot be wished
away so we better acknowledge them (diffusion is somewhat better unevent
gradient scenarios but @ cost of slower convergence under normal condition
pi times eye measure ;-).

A possible partial remediation to the problem is to prioritize flooding of
topology information over reachability (prefixes) but alas, in ISIS that's
not how the packets are laid out.

back to shelling peanuts ;-)

--- tony

On Wed, May 6, 2020 at 11:19 AM Joel M. Halpern <[email-masked]> wrote:

> flooding in the presence of a node which does not support it. Your options
> to reduce the period of looping will be:
> >
> >>>> [Bruno] The example is straightforward but you are referring to FIB
> and IP
> >>>> D has now received 600 of the 1000 LSP updates. Still no changes to
> its
> >> forwarding plane.
> >>>> When all links are up, shortest path for all northbound destinations
> is via
> >> C.
> >>>> Let’s say fast flooding is 500 LSPs/second and slow flooding (Node D)
> is 20
> >> LSPs/seconds.
> >>>>> >From t0, the LSDB is inconsistent across the network as all nodes
> but k
> >> are
> >>>>> - for all nodes across the network to learn about the LSP, and get
> in sync
> >> with
> >>>>>> I think it is also useful to note that even the simple test case
> which
> >> Bruno
> >>>>> think is a great strength of the protocol - particularly in
> environments
> >> where
> >>>>> both of the draft proposals discuss this - though I agree more
> detail will
> >> be
> >>>>> required.
> >>>>>> It is intuitive that if you want to flood faster you also need to
> ACK
> >> faster -
> >>>>>> It is also true - as you state - that sending ACKs more quickly
> will result
> >> in
> >>>>> isis-flooding-scale has been silent on this - but not because we
> aren't
> >> aware
> >>>>>> In summary, I don't expect to have to "reinvent TCP" - but I do
> think
> >> you
> >>>>> pas etre diffuses, exploites ou copies sans autorisation. Si vous
> avez
> >> recu ce
> >> deforme ou
> >>>>> If you have received this email in error, please notify the sender
> and
> >> delete

---

From: Mitchell Erblich <[email-masked]>
Date: Wed, 6 May 2020 11:57:52 -0700

Robert,

		I agree with you…

		Disruptions based on protocol changes with non-current/legacy type products and/or intenal black-hole , loss of packets/rexmits, aka reliability should be primary.  Even good/great ideas of IPv4 / IPv6 intermediate systems/routers find small incompatibilities either with the protocol and/or CLI errors.

		With that said, I definitely like the OSPF DR/BDR functionality in Eth/multi-access/non-p2p environments for single router failures IMO adds reliability and mitigates certain types of disruptions.

		However, with wireless devices, one must still respect that bottlenecks need to be resolved when dealing with these slower/down ward scaling environments. This is just not just a packet-per-sec / bytes-per-second issue but frequently are smaller packet/segment sizes , a scaling of the number of simultaneous connections, and loss of packets/segments in these newer environments

		And, sorry, Eth is there with 100Gb and LACP exists, that will eventually push upward scaling proposals.

Mitchell Erblich
[email-masked]

> On May 6, 2020, at 12:52 AM, Robert Raszuk <[email-masked]> wrote:
> 
> Christian,
> 
> It is about impact to the overall network service when you make ISIS convergence not consistent across nodes of the topology, I think our goal is not to converge ISIS faster by improving flooding speed .... it is much more about how to deliver user packets with minimal disruption upon topology changes. 
> 
> Best,
> R.
>

---

From: Jeff Tantsura <[email-masked]>
Date: Wed, 6 May 2020 21:41:38 -0700

Robert,

Assuming C and E provide access to the same set of destinations, that are closer of further away from C and E.
B (which is fast), after it notifies A that it can’t reach C directly will cause A to send traffic to D. D - dependent on total cost would start happily sending some traffic towards destinations behind C to B so LFA on B wouldn’t really help.

Cheers,
Jeff
On May 5, 2020, 5:04 PM -0700, Robert Raszuk <[email-masked]>, wrote:
> Hi Les,
>
> A side comment but your example shows another - one may say even much more serious issue.
>
> Assume we have LFA/TI-LFA enabled in the network and precomputed on B which get's activated and shifts traffic to E when detects that C is down. Detection is fast .. 10s-100s of milliseconds.
>
> Now if B converges fast and recomputes topology much faster then D it may remove protection and send packets to D natively. Well clearly as we established D is slow and will loop it back.
>
> That is why I mentioned the other day that a fast control plane is not always a good thing (I am sure many will say the opposite - but it is ok ;).
>
> > On Wed, May 6, 2020 at 1:35 AM Les Ginsberg (ginsberg) <ginsberg=[email-masked]> wrote:
> > > Bruno -
> > >
> > > >         > action for transmission of Partial Sequence Number PDUs.
> > > >         > It shall be less than minimumLSPTransmission-Interval."
> > > > >
> > > > implementations I am aware of use values in this order of magnitude.. These
> > > >     > Les
> > > > > > in the TCP acks, how much data it can or wants to receive..
> > > > > > send a PSNP when it receives a LSP.. 1) the receiver should jitter the
> > > > > > PSNP,
> > > > > > There is no ordering when flooding LSPs, or sending PSNPs.. This makes
> > > > > > sure they send slow enough to not overload the slowest IS.. I'm not sure

---

From: <[email-masked]>
Date: Thu, 7 May 2020 09:17:31 +0000

Les,

> I am sorry it has been so difficult for us to understand each other. I am trying my best.

+1 +1

> I am the vendor.

I'm not sure what (technical) point you are trying to make.

Coming back to your statement : “when only some nodes in the network support faster flooding […]  it prolongs the period of LSDB inconsistency.”

Your example is flawed on two points:

- Your performance indicator is "micro loop duration", while we are talking about the duration of LSP flooding across the network. So the metric should be "LSP flooding time" (or "period of LSDB inconsistency")

- Your example is about one node doing slower flooding, while we are interested in the case when one node support faster flooding. (It's quite clear that if one node is slower, the end to end flooding time may be longer. The point you are raising is when one node is doing faster flooding)

Can you fix your example on those 2 points?

Thanks

--Bruno

> > > > [Bruno] The example is straightforward but you are referring to FIB and IP

> > > > Consider the state of Node B and Node D at various time points from the

> > > > B has completed FIB updates. Traffic to all Northbound destinations is

> > being forwarded via D.

> > > > D has now received 600 of the 1000 LSP updates. Still no changes to its

> > > > Seems like it was not too long ago that we were discussing this in person.

> > > > There is a topology northbound of C and E (not shown) and a topology

> > southbound of A (not shown).

> > > > When all links are up, shortest path for all northbound destinations is via

> > > > Let’s say fast flooding is 500 LSPs/second and slow flooding (Node D) is 20

> > > > All nodes except for D can receive these in 2 seconds (plus internode

> > delay time).

> > > > > > when only some nodes in the network support faster flooding the

> > behavior

> > > > > of the whole network may not be "better" when faster flooding is

> > enabled

> > > > > 2) If not, can you provide an example where increasing the flooding

> > rate on

> > > > > > when only some nodes in the network support faster flooding the

> > behavior

> > > > > of the whole network may not be "better" when faster flooding is

> > enabled

> > > > > 2) If not, can you provide an example where increasing the flooding

> > rate on

> > > > > 3) In the meantime, let's try the theoretical analysis on a simple

> > scenario

> > > > > - Let's call Dij the time needed to flood the LSP from node i to the

> > adjacent

> > > > > >From t0, the LSDB is inconsistent across the network as all nodes but k

> > > > > - for all nodes across the network to learn about the LSP, and get in sync

> > > > > Then how can reducing the flooding delay on one adjacency could

> > prolongs

> > > > > It seems to me that it can only improve/decrease it. Otherwise, this

> > would

> > > > > mean that decreasing the cost on a link can increase the cost of the

> > shortest

> > > > > Note: I agree that there are other cases, such as  multiple LSPs

> > originated by

> > > > > the same node, and multiple LSPs originated by multiple nodes, but

> > let's start

> > > > > > I have read your later posts on this thread as well - but decided to

> > reply to

> > > > > > There are now two proposals as to how to address the issue - neither

> > > > > > I have commented on why IS-IS flooding requirements are

> > significantly

> > > > > > I think it is also useful to note that even the simple test case which

> > > > > reported on in last week's interim meeting demonstrated that without

> > > > > changes to the protocol at all IS-IS was able to flood an order of

> > magnitude

> > > > > > This gives me hope that we are looking at the problem correctly and

> > will not

> > > > > > b)Removal of the independence of the IS-IS protocol from the

> > address

> > > > > families whose reachability advertisements it supports - something

> > which I

> > > > > think is a great strength of the protocol - particularly in environments

> > > > > > Your comments regarding PSNP response times are quite correct -

> > > > > both of the draft proposals discuss this - though I agree more detail will

> > > > > > It is intuitive that if you want to flood faster you also need to ACK

> > > > > > The basic relationship between retransmit interval and PSNP interval

> > is

> > > > > implementations I am aware of use values in this order of magnitude.

> > These

> > > > > numbers need to be reduced if we are to flood faster, but the

> > relationship

> > > > > > It is also true - as you state - that sending ACKs more quickly will result

> > > > > additional PDUs which need to be received/processed by IS-IS - and this

> > > > > some impact. But I think it is reasonable to expect that an

> > implementation

> > > > > which can support sending and receiving LSPs at a faster rate should

> > also be

> > > > > able to send/receive PSNPs at a faster rate. But we still need to be

> > smarter

> > > > > > LANs are a more difficult problem than P2P - and thus far draft-

> > ginsberg-lsr-

> > > > > isis-flooding-scale has been silent on this - but not because we aren't

> > > > > > What the best behavior on a LAN may be is something I am still

> > considering.

> > > > > Slowing flooding down to the speed at which the slowest IS on the LAN

> > > > > support may not be the best strategy - as it also slows down the

> > propagation

> > > > > rate for systems downstream from the nodes on the LAN which can

> > handle

> > > > > faster flooding - thereby having an impact on flooding speed

> > throughout the

> > > > > network in a way which may be out of proportion. This is a smaller

> > example

> > > > > of the larger issue that when only some nodes in the network support

> > > > > flooding the behavior of the whole network may not be "better" when

> > > > > flooding is enabled because it prolongs the period of LSDB

> > > > > > In summary, I don't expect to have to "reinvent TCP" - but I do think

> > > > > have provided a useful perspective for us to consider as we progress on

> > > > > > > Subject: [Lsr] Why only a congestion-avoidance algorithm on the

> > sender

> > > > > > > Two years ago, Gunter Van de Velde and myself published this

> > draft:

> > > > > > > My thoughts were that once we start implementing new algorithms

> > > > > > > optimize ISIS flooding speed, we'll end up with our own version of

> > TCP.

> > > > > > > overloaded. The receiver indicates, through the receiver-window-

> > size

> > > > > > > 2) Congestion control is: the sender trying to prevent the links

> > between

> > > > > > > how the transmission is behaving. Are there drops ? What is the RTT

> > > > > > > Do drop-percentage and RTT change ? Do acks come in at the same

> > rate

> > > > > > > as the sender sends segments ? Are there duplicate acks ? To be

> > able

> > > > > > > If you want an ISIS sender to make a guess at what speed it can

> > send,

> > > > > > > without changing the protocol, the only thing the sender can do is

> > look

> > > > > > > at the PSNPs that come back from the receiver. But the RTT of

> > PSNPs can

> > > > > > > In TCP, if a single segment gets lost, each new segment will cause

> > > > > > > a sender can get from looking at incoming PSNPs is a lot less than

> > what

> > > > > > > And I think most implementations use that as the default. This

> > means

> > > > > > > they want. This means a sender can not really make conclusions

> > about

> > > > > > > There is no ordering when flooding LSPs, or sending PSNPs. This

> > makes

> > > > > > > full 5 seconds before it can make any real conclusions about

> > dropped

> > > > > > > not be able to react without a delay of a few seconds. A sender

> > might

> > > > > > > hunderds or thousands of LSPs in those 5 seconds, which might all

> > or

> > > > > > > algorithm for the sender. We must also specify for the receiver a

> > more

> > > > > > > Note that this might result in the receiver sending more (and

> > smaller)

> > > > > > > 2) more detailed algorithm on receiver when and how to send

> > > > > > > interfaces, linecards, etc). That's implementation dependent. A

> > receiver

> > > > > > > can definitely advertise a fixed value. So the sender has an upper

> > bound

> > > > > > > receiver can even advertise a dynamic value. Maybe now, maybe

> > only in

> > > > > > > The problem I saw with flooding over TCP concerns multi-point

> > networks

> > > > > > > When flooding over a multi-point network, setting up TCP

> > connections

> > > > > > > Full mesh ? Or do all ISes on a LAN create a TCP-connection to the

> > DIS ?

> > > > > > > There is no backup DIS in ISIS (unlike OSPF). Things get messy

> > quickly.

> > > > > > > there were drops ? There are no acks (PSNPs) on a LAN. We assume

> > > > > > > that are broadcasted are received by all other ISes on the LAN.

> > There

> > > > > > > PSNPs to request retransmissions. It seems impossible (or very

> > hard) to

> > > > > > > me for all ISes on a LAN to keep track of dropped LSPs and adjust

> > their

> > > > > > > all ISes can see what the lowest advertised sending-speed is. And

> > make

> > > > > > > sure they send slow enough to not overload the slowest IS. I'm not

> > sure

> > > > > > > Sender-side congestion-control won't work without specifying in

> > more

> > > > > > > good enough. I don't know if advertising a static value is good

> > enough.

> > > > > pas etre diffuses, exploites ou copies sans autorisation. Si vous avez

> > > > > If you have received this email in error, please notify the sender and

> > > > > As emails may be altered, Orange is not liable for messages that have

> > been

> > > > Orange decline toute responsabilite si ce message a ete altere, deforme

> > > > If you have received this email in error, please notify the sender and

> > delete this message and its attachments.

> > > > As emails may be altered, Orange is not liable for messages that have

> > been modified, changed or falsified.

> > > > Orange decline toute responsabilite si ce message a ete altere, deforme

> > > > If you have received this email in error, please notify the sender and

> > delete this message and its attachments.

> > > > As emails may be altered, Orange is not liable for messages that have

> > been modified, changed or falsified.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 7 May 2020 11:20:31 +0200

Hey Jeff,

What if in your analysis A-D link metric is 1500 ? Still A will send to B !

See in all of this discussion from my pov it is not really about danger of
making LSDB inconsistent for much longer.

It is however about operator upgrading to new release a router, maybe even
reading release notes stating that flooding speed optimizations defined in
RFC ZZZ are implemented and final effect he sees are microloops lasting 10s
of seconds for his traffic.

Yes some vendors have good tools to protect from microloops, but those must
be enabled. Moreover those still require manual timer configuration how
long to keep loop free protection in place. And I am not aware any vendor
has a cli command showing how long it took in all routers in my area/level
to install in FIB result of various levels/cases of topology changes.

So I am not saying faster is not good. I am saying if you make car faster
do not forget to equip it with seat belts and make enough of warnings that
driver will actually fasten them.

Best,
R.

On Thu, May 7, 2020 at 6:41 AM Jeff Tantsura <[email-masked]>
wrote:

> B (which is fast), after it notifies A that it can’t reach C directly will
> cause A to send traffic to D. D - dependent on total cost would start
> happily sending some traffic towards destinations behind C to B so LFA on B
> wouldn’t really help.
>
> Cheers,
> Jeff
>

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Thu, 7 May 2020 14:55:28 +0000

Bruno –

I have specifically used an example where “microloop avoidance” is not applicable. So I did not want to use the term “microloop” but rather used “loop” so as not to suggest that “microloop avoidance” is a potential solution for the sub-optimal behavior.
Hope you can appreciate that point.

It would be easy enough to include more nodes in the topology which only support slow flooding. The end result would be the same.
I have kept the example simple in the hopes we could more easily agree that what I describe can happen when not all nodes support faster flooding – which is the only point I am trying to make. Whether the ratio of Fast/Slow nodes is large or small or about the same doesn’t eliminate the possibility that the same behavior could be seen – though it might alter the location of the topology change which would be problematic.

From an operator’s POV, I am pretty sure that what you really care about is whether packets get successfully forwarded or not. I am demonstrating that it isn’t safe to assume forwarding behavior will be optimal when not all nodes support fast flooding.

   Les

From: [email-masked] <[email-masked]>
Sent: Thursday, May 07, 2020 2:18 AM
To: Les Ginsberg (ginsberg) <[email-masked]>
Cc: [email-masked]; Christian Hopps <[email-masked]>
Subject: RE: [Lsr] Flooding across a network

Les,

+1 +1

Thanks

--Bruno

> > To: Christian Hopps <[email-masked]<mailto:[email-masked]>>

> > Cc: Les Ginsberg (ginsberg) <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>

> > > > On May 6, 2020, at 10:33 AM, [email-masked]<mailto:[email-masked]> wrote:

> > > > To: DECRAENE Bruno TGI/OLN; lsr@ietf..org<mailto:lsr@ietf..org>

Thank you.

---

From: [email-masked]
Date: Thu, 7 May 2020 08:52:43 -0700

Les,

> From an operator’s POV, I am pretty sure that what you really care about is whether packets get successfully forwarded or not. I am demonstrating that it isn’t safe to assume forwarding behavior wi…

Your point is well-made. There are certainly going to be topologies where making flooding faster would result in worse behavior.

However, the point is that there are also topologies where flooding faster would improve convergence. To support those, we must have faster flooding. Please focus on that goal.

Regards,
Tony

---

From: <[email-masked]>
Date: Thu, 7 May 2020 16:02:32 +0000

Les,

From: Les Ginsberg (ginsberg) [mailto:[email-masked]]
Sent: Thursday, May 7, 2020 4:55 PM
To: DECRAENE Bruno TGI/OLN
Cc: [email-masked]; Christian Hopps
Subject: RE: [Lsr] Flooding across a network

Bruno –

I have specifically used an example where “microloop avoidance” is not applicable. So I did not want to use the term “microloop” but rather used “loop” so as not to suggest that “microloop avoidance” is a potential solution for the sub-optimal behavior.
[Bruno] I’m not sure what you mean exactly by “microloop avoidance”.
On my side, I mean “Loop avoidance using Segment Routing” [1] which _is_ applicable. (Note that I’m not saying that all _implementations_ cover all cases.)
[1]  https://tools.ietf.org/html/draft-bashandy-rtgwg-segment-routing-uloop-08

It would be easy enough to include more nodes in the topology which only support slow flooding. The end result would be the same.
[Bruno] The question is whether the activation of fast flooding on one (some) node(s)/IGP adjacency(es) may result in delaying the LSDB synchronization network wide. It’s not about the number of slow nodes.

You seem to assume that we have a majority of fast nodes, and some remaining slow nodes. In which cases I agree that in some cases the overall/network wide behavior may be a slow LSDB sync. But not slower than with slow nodes only.
I have kept the example simple in the hopes we could more easily agree that what I describe can happen when not all nodes support faster flooding – which is the only point I am trying to make.
[Bruno] If your point is that while we have slow nodes in the network, in some cases the network wide behavior can be “slow”, somewhere between “all nodes are fast” and “all nodes are slow”, then I agree with you.

However what you said is: “when only some nodes in the network support faster flooding […]  it prolongs the period of LSDB inconsistency.”
In that sentence, I disagree with “prolongs” which according to 2 dictionaries means “lasting longer”.
https://dictionary.cambridge.org/fr/dictionnaire/anglais/prolong
https://www.merriam-webster.com/dictionary/prolong

Because, at least to me, this reads as “The introduction of fast-flooding nodes in the network (may) increase the period of LSDB inconsistency.”

Whether the ratio of Fast/Slow nodes is large or small or about the same doesn’t eliminate the possibility that the same behavior could be seen – though it might alter the location of the topology change which would be problematic.

From an operator’s POV, I am pretty sure that what you really care about is whether packets get successfully forwarded or not.
[Bruno] Yes, but let’s not combine multiple (complex) problems.
Here I’d like we focus on LSP flooding and LSDB synchronization across the network. Plus this is exactly the point that you raised: “increase the period of LSDB inconsistency.”

I am demonstrating that it isn’t safe to assume forwarding behavior will be optimal when not all nodes support fast flooding.
[Bruno] Which is very different from your original point “when only some nodes in the network support faster flooding […]  it prolongs the period of LSDB inconsistency.”

--Bruno

   Les

Les,

+1 +1

Thanks

--Bruno

Thank you.

---

From: Gyan Mishra <[email-masked]>
Date: Sun, 17 May 2020 05:06:51 -0400

Am reading through this thread late but want to chime in on the discussion.

Router Isis
fast-flood x

The concept of ISIS fast flooding has been around for decades and is
critical for ISIS LSDB synchronization to so all nodes have all prefixes to
avoid micro loops and within the network.

The concept of fast flood feature is basically to flood the number of LSPs
that need to be flooded before starting the local SPF.  So with fast flood
enabled all nodes in the domain wait till the last LSP is flooded before it
runs its local SPF.  By fast flooding it reduces the overall number of SPFs
executed and allows the LSDB to be synchronized by all nodes in the domain.

Let’s say a link went down or came up topology change.  The router should
at a minimum flood at least that LSP that triggered the SPF before running
its local SPF.  Fast flooding is recommended by all vendors as it improves
convergence times and limits duration of LSDB inconsistencies during which
time  micro loops form.

In case where a few routers don’t have fast flood enabled, those  few
router in slow mode may have to run multiple SPFs before completely being
synchronized.  I think that question where a mix of slow and fast exist
versus all slow or all fast you may have mixed unpredictable behaviors that
you really would have to test out in a lab or live network.

In my experience I have always had all nodes running fast flood mode as the
feature has been around for a long time and recommended for optimal
convergence.

There are many other parameters with ISIS that come into play to optimize
convergence below which may vary by vendor but can improve convergence and
minimize micro loops below:

One of them is setting all P2P routed links to circuit type point-to-point
to avoid DIS election.

Both ISIS and OSPF have the iSPF incremental SPF so only the changed part
of the tree is updated and not the entire tree to save on SPF processing.

With short spf interval and spf delay the local spf May start before the
lsp that triggered the spf is flooded.

Lsp pacing can be used to speed up end to end flooding.

Hello padding TLV 8 for MTU defection to help with convergence so max
number of LSPs can be sent during flooding.

Increase lsp lifetime to maximum to reduce control traffic so cpu can be
optimized to refresh lsp’s

Reduce the frequency of periodic lsp flooding of topology to reduce link
utilization by Isis

Ignore lsp errors instead of purging them

Carful balance of event processing and throttling of events to maximize
convergence times

Prefix prioritization on critical prefixes such as MPLS FEC binding.

LFA and RLFA for pre computed backup paths

BFD for ms failure detection

Overall Isis as far as scalability far exceeds ospf with stability with
domains with a very large number of nodes in the 100s and high number of
adjacencies per node.

As far as stability and convergence both ospf and Isis are equivalent from
my experience.

Kind regards

Gyan
Verizon

On Thu, May 7, 2020 at 12:03 PM <[email-masked]> wrote:

> Les,
>
>
>
>
>
> *From:* Les Ginsberg (ginsberg) [mailto:[email-masked]]
> *Sent:* Thursday, May 7, 2020 4:55 PM
> *To:* DECRAENE Bruno TGI/OLN
> *Cc:* [email-masked]; Christian Hopps
> *Subject:* RE: [Lsr] Flooding across a network
>
>
>
> Bruno –
>
>
>
> I have specifically used an example where “microloop avoidance” is not
> applicable. So I did not want to use the term “microloop” but rather used
> “loop” so as not to suggest that “microloop avoidance” is a potential
> solution for the sub-optimal behavior.
>
> On my side, I mean “Loop avoidance using Segment Routing” [1] which _*is*_
> applicable. (Note that I’m not saying that all _*implementations*_ cover
> all cases.)
>
> [1]
> https://tools.ietf.org/html/draft-bashandy-rtgwg-segment-routing-uloop-08
>
>
>
>
>
> It would be easy enough to include more nodes in the topology which only
> support slow flooding. The end result would be the same.
>
> [Bruno] The question is whether the activation of fast flooding on one
> (some) node(s)/IGP adjacency(es) may result in delaying the LSDB
> synchronization network wide. It’s not about the number of slow nodes.
>
>
>
> You seem to assume that we have a majority of fast nodes, and some
> remaining slow nodes. In which cases I agree that in some cases the
> overall/network wide behavior may be a slow LSDB sync. But not slower than
> with slow nodes only.
>
> I have kept the example simple in the hopes we could more easily agree
> that what I describe can happen when not all nodes support faster flooding
> – which is the only point I am trying to make.
>
> [Bruno] If your point is that while we have slow nodes in the network, in
> some cases the network wide behavior can be “slow”, somewhere between “all
> nodes are fast” and “all nodes are slow”, then I agree with you.
>
>
>
> However what you said is: “when only some nodes in the network support
> Because, at least to me, this reads as “The introduction of fast-flooding
> nodes in the network (may) increase the period of LSDB inconsistency.”
>
>
>
> Whether the ratio of Fast/Slow nodes is large or small or about the same
> doesn’t eliminate the possibility that the same behavior could be seen –
> though it might alter the location of the topology change which would be
> problematic.
>
>
>
> From an operator’s POV, I am pretty sure that what you really care about
> is whether packets get successfully forwarded or not.
>
> Here I’d like we focus on LSP flooding and LSDB synchronization across the
> network. Plus this is exactly the point that you raised: “increase the
> period of LSDB inconsistency.”
>
>
>
> I am demonstrating that it isn’t safe to assume forwarding behavior will
> be optimal when not all nodes support fast flooding.
>
> [Bruno] Which is very different from your original point “when only some
> nodes in the network support faster flooding […]  it prolongs the period
> of LSDB inconsistency.”
>
>
>
> --Bruno
>
>
>
>    Les
>
>
>
>
>
> *From:* [email-masked] <[email-masked]>
> *Sent:* Thursday, May 07, 2020 2:18 AM
> *To:* Les Ginsberg (ginsberg) <[email-masked]>
> *Cc:* [email-masked]; Christian Hopps <[email-masked]>
> *Subject:* RE: [Lsr] Flooding across a network
>
>
>
> Les,
>
>
>
> > From: Les Ginsberg (ginsberg) [mailto:[email-masked]
> <[email-masked]>]
>
> >
>
> > Bruno -
>
> >
>
> Coming back to your statement : “when only some nodes in the network
> support faster flooding […]  it prolongs the period of LSDB inconsistency
> .”
>
> - Your performance indicator is "micro loop duration", while we are
> talking about the duration of LSP flooding across the network. So the
> metric should be "LSP flooding time" (or "period of LSDB inconsistency")
>
> - Your example is about one node doing slower flooding, while we are
> interested in the case when one node support faster flooding. (It's quite
> clear that if one node is slower, the end to end flooding time may be
> longer. The point you are raising is when one node is doing faster flooding)
>
>
>
> > > > From: Christian Hopps [mailto:[email-masked] <[email-masked]>]
>
> > > >
>
> > > > > From: Les Ginsberg (ginsberg) [mailto:[email-masked]
> <[email-masked]>]
>
> > > > > From: Les Ginsberg (ginsberg) [mailto:[email-masked]
> <[email-masked]>]
>
> inconsistency.”
> > > > > > > From: Lsr [mailto:[email-masked] <[email-masked]>]
> On Behalf Of Les Ginsberg
>
> > > > > > (ginsberg)
>
-- 

Gyan  Mishra

Network Engineering & Technology

Verizon

Silver Spring, MD 20904

Phone: 301 502-1347

Email: [email-masked]

---

From: <[email-masked]>
Date: Mon, 18 May 2020 12:42:13 +0000

Hello Gyan,

Nice to know that fast LSP flooding is important to you.
You are right that there are existing code and knobs to improve IGP convergence in general, and in particular to the specific implementation you are referring to (I’d bet that its first letter start with C).
Focusing on efficient flooding, which is the subject of this thread, “fast-flood x” is indeed a nice name. AFAIR, the feature has been introduced approximately 15 years ago and was indeed an improvement. Many more have been introduced as part of the fast IGP convergence project that I’d been running in this company, resulting in very significant and much needed improvement on that implementation.
Despite the name “fast flood”, I don’t think that anyone is claiming that it’s solving the broad issue of efficient LSP flooding.
LSP-pacing may also be configured in order to influence the delay between two consecutively sent LSP.

One question, which you have likely faced when enabling these knobs, is the right values to configure, depending on the receiver (’s capability).
If you have multiple implementation on the sender side –with different ways to implement ‘fast flooding’- and different platforms (‘s capability) on the receiver side, combined with various usages (e.g. from 2 to 50 IGP adjacencies, with or without BGP/RSVP-TE/multicast….) the matrix is quickly getting significant. Your vendor of the receiver implementation may help you; or not (significantly).
With IS-IS, the problem is exacerbated because the protocol is typically vital to the network and all its usages and customers. So nobody wants to be held accountable if the parameters turned to be ‘non-optimal’ over the next decade. Especially as routing ‘incidents’ may already happen with default configuration.

Sending fast is “easy”, but what we need is an effective communication between the two entities involved: one sender and a receiver; sometimes referred to as goodput. RFC 2914 “Congestion Control Principles” [1] may be an introduction to this.
e.g. you can probably speak English to me at 200-250 words per minutes but I can assure you that it would be more effective to speak to me at a lower rate, because of limitations on the receiver side.
That’s similar for machines in general and IS-IS in particular, especially since IS-IS is not that fast to recover from lost LSPs.
The problem is not new and referred to as flow control. From [2] “In data communications<https://en.wikipedia.org/wiki/Data_communications>, flow control is the process of managing the rate of data transmission between two nodes to prevent a fast sender from overwhelming a slow receiver. It provides a mechanism for the receiver to control the transmission speed, so that the receiving node is not overwhelmed with data from transmitting node.”

That’s what we are working on.

Kind regards,
Bruno

[1] https://tools.ietf.org/html/rfc2914
[2] https://en.wikipedia.org/wiki/Flow_control_(data)

From: Gyan Mishra [mailto:[email-masked]]
Sent: Sunday, May 17, 2020 11:07 AM
To: DECRAENE Bruno TGI/OLN
Cc: Christian Hopps; Les Ginsberg (ginsberg); [email-masked]
Subject: Re: [Lsr] Flooding across a network

Router Isis
fast-flood x

The concept of ISIS fast flooding has been around for decades and is critical for ISIS LSDB synchronization to so all nodes have all prefixes to avoid micro loops and within the network.

The concept of fast flood feature is basically to flood the number of LSPs that need to be flooded before starting the local SPF.  So with fast flood enabled all nodes in the domain wait till the last LSP is flooded before it runs its local SPF.  By fast flooding it reduces the overall number of SPFs executed and allows the LSDB to be synchronized by all nodes in the domain.

Let’s say a link went down or came up topology change.  The router should at a minimum flood at least that LSP that triggered the SPF before running its local SPF.  Fast flooding is recommended by all vendors as it improves convergence times and limits duration of LSDB inconsistencies during which time  micro loops form.

In case where a few routers don’t have fast flood enabled, those  few router in slow mode may have to run multiple SPFs before completely being synchronized.  I think that question where a mix of slow and fast exist versus all slow or all fast you may have mixed unpredictable behaviors that you really would have to test out in a lab or live network.

In my experience I have always had all nodes running fast flood mode as the feature has been around for a long time and recommended for optimal convergence.

There are many other parameters with ISIS that come into play to optimize convergence below which may vary by vendor but can improve convergence and minimize micro loops below:

One of them is setting all P2P routed links to circuit type point-to-point to avoid DIS election.

Both ISIS and OSPF have the iSPF incremental SPF so only the changed part of the tree is updated and not the entire tree to save on SPF processing.

With short spf interval and spf delay the local spf May start before the lsp that triggered the spf is flooded.

Hello padding TLV 8 for MTU defection to help with convergence so max number of LSPs can be sent during flooding.

Increase lsp lifetime to maximum to reduce control traffic so cpu can be optimized to refresh lsp’s

Reduce the frequency of periodic lsp flooding of topology to reduce link utilization by Isis

Carful balance of event processing and throttling of events to maximize convergence times

Overall Isis as far as scalability far exceeds ospf with stability with domains with a very large number of nodes in the 100s and high number of adjacencies per node.

As far as stability and convergence both ospf and Isis are equivalent from my experience.

Kind regards

Gyan
Verizon

On Thu, May 7, 2020 at 12:03 PM <[email-masked]<mailto:[email-masked]>> wrote:
Les,

From: Les Ginsberg (ginsberg) [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Thursday, May 7, 2020 4:55 PM
To: DECRAENE Bruno TGI/OLN
Cc: [email-masked]<mailto:[email-masked]>; Christian Hopps
Subject: RE: [Lsr] Flooding across a network

Bruno –

--Bruno

   Les

Cc: [email-masked]<mailto:[email-masked]>; Christian Hopps <[email-masked]<mailto:[email-masked]>>

Les,

+1 +1

Thanks

--Bruno

--
Gyan  Mishra
Network Engineering & Technology
Verizon
Silver Spring, MD 20904
Phone: 301 502-1347
Email: [email-masked]<mailto:[email-masked]>

---

From: Gyan Mishra <[email-masked]>
Date: Tue, 19 May 2020 02:18:50 -0400

On Mon, May 18, 2020 at 8:42 AM <[email-masked]> wrote:

> You are right that there are existing code and knobs to improve IGP
> convergence in general, and in particular to the specific implementation
> you are referring to (I’d bet that its first letter start with C).
>
> Focusing on efficient flooding, which is the subject of this thread,
> “fast-flood x” is indeed a nice name. AFAIR, the feature has been
> introduced approximately 15 years ago and was indeed an improvement. Many
> more have been introduced as part of the fast IGP convergence project that
> I’d been running in this company, resulting in very significant and much
> needed improvement on that implementation.
>
> Despite the name “fast flood”, I don’t think that anyone is claiming that
> it’s solving the broad issue of efficient LSP flooding.
>
> One question, which you have likely faced when enabling these knobs, is
> the right values to configure, depending on the receiver (’s capability)
>
If you have multiple implementation on the sender side –with different ways
> to implement ‘fast flooding’- and different platforms (‘s capability) on
> the receiver side, combined with various usages (e.g. from 2 to 50 IGP
> adjacencies, with or without BGP/RSVP-TE/multicast….) the matrix is quickly
> getting significant. Your vendor of the receiver implementation may help
> you; or not (significantly).
>
   Gyan> I completely understand the issue and that makes sense that with
the variables thrown out there with a mixed vendor environment with a
variety of platforms and sender flooding capability, and the receiver
receiving the flooded LSP capability to be processed during its local SPF
and overrunning the receivers.  So this is a real world worst case scenario
which can happen with flow control and how to deal with the resulting slow
convergence and sub optimal routing with micro loops. Got it.

Thank you for the explanation.

> With IS-IS, the problem is exacerbated because the protocol is typically
> vital to the network and all its usages and customers. So nobody wants to
> be held accountable if the parameters turned to be ‘non-optimal’ over the
> next decade. Especially as routing ‘incidents’ may already happen with
> default configuration.
>
>  Gyan>  Default change are painful for operators but unfortunately they do
> happen and throw off any tuning or standard template deployed.
>
>
>
> Sending fast is “easy”, but what we need is an effective communication
> between the two entities involved: one sender and a receiver; sometimes
> referred to as goodput. RFC 2914 “Congestion Control Principles” [1] may
> be an introduction to this.
>
> e.g. you can probably speak English to me at 200-250 words per minutes but
> I can assure you that it would be more effective to speak to me at a lower
> rate, because of limitations on the receiver side.
>
> That’s similar for machines in general and IS-IS in particular, especially
> since IS-IS is not that fast to recover from lost LSPs.
>
> The problem is not new and referred to as flow control. From [2] “In data
> communications <https://en.wikipedia.org/wiki/Data_communications>, *flow
> control* is the process of managing the rate of data transmission between
> two nodes to prevent a fast sender from overwhelming a slow receiver. It
> provides a mechanism for the receiver to control the transmission speed, so
> that the receiving node is not overwhelmed with data from transmitting
> node.”
>
>
>
>  Gyan>  Makes sense.
> *From:* Gyan Mishra [mailto:[email-masked]]
> *Sent:* Sunday, May 17, 2020 11:07 AM
> *Cc:* Christian Hopps; Les Ginsberg (ginsberg); [email-masked]
> *Subject:* Re: [Lsr] Flooding across a network
>
>
>
>
>
> In my experience I have always had all nodes running fast flood mode as
> the feature has been around for a long time and recommended for optimal
> convergence.
>
>
>
> nodes in the network support faster flooding […]  it prolongs the period of
> LSDB inconsistency.”
>
>
>
> --Bruno
>
>
>
>    Les
>
>
>
>
>

<http://www.verizon.com/>

*Gyan Mishra*

*Network Solutions A**rchitect *

*M 301 502-134713101 Columbia Pike *Silver Spring, MD