Subject: [Lsr] IS-IS over TCP

From: [email-masked]
Date: Tue, 6 Nov 2018 10:51:33 +0700

Per the WG meeting, discussing on the list:

This is good work and I support it.

I would remind folks that TCP is NOT the only transport protocol available and that perhaps we should be considering QUIC while we’re at it.  In particular, flooding is a (relatively) low bandwidth operation in the modern network and we could avoid slow-start issues by using QUIC.

Tony

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 5 Nov 2018 23:20:56 -0500

> This is good work and I support it.

Ditto.

> I would remind folks that TCP is NOT the only transport protocol available
> and that perhaps we should be considering QUIC while we’re at it.  In
> particular, flooding is a (relatively) low bandwidth operation in the
> modern network and we could avoid slow-start issues by using QUIC.

I'm ambivalent of the transport, but agree that TCP shouldn't be the default
answer.

My concerns that I tried raising via jabber summarize roughly as follows:
- TCP is prone to interesting backpressure issues, typically as a result of
  packet loss or slow receivers.
- TCP timers can react poorly in some environments where you may want time
  sensitive things.  This includes something as long as 3 second BGP hold
  timers.
- IGPs have a lot of interesting timer hacks to try to ensure that a given
  domain has a consistent database prior to running an SPF.  In the face of
  "stuck" flooding due to backpressure or other things, some of these may need
  to be revisited.

It's been over a year since I looked at QUIC.  I agree with Tony that a
number of the properties it had on my last read are desirable.  I'd suggest
that its behavior (especially timers) in the event of packet loss should be
given a look at based on the comments above.

- Jeff

---

From: Henk Smit <[email-masked]>
Date: Tue, 06 Nov 2018 05:21:43 +0100

Thanks, Tony.

We picked TCP because every router on the planet already has a TCP stack 
in it.
That made it the obvious choice.

Our draft described a TVL in the IIHs to indicate a router's
ability to use TCP for flooding.
That TLV has several sub-TVLs.
1) the TCP port-number
2) an IPv4 address
3) and/or an IPv6 address

We can change the first sub-TVL so that it indicates:
1) 1 or 2 bytes indicating what protocol to use
2) the remainder of the sub-TLV is an indicator what port-number
    or other identifier to use to connect over that protocol.

This way we can start improving IS-IS with TCP today.
And add/replace it with other protocols in the future.

henk.

[email-masked] schreef op 2018-11-06 04:51:
> I would remind folks that TCP is NOT the only transport protocol
> available and that perhaps we should be considering QUIC while we’re
> at it.  In particular, flooding is a (relatively) low bandwidth
> operation in the modern network and we could avoid slow-start issues
> by using QUIC.
> 
> Tony
> 
> _______________________________________________
> Lsr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lsr

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Wed, 7 Nov 2018 04:09:44 +0000

Henk/Gunter -

Couple of points.

1)IS-IS PDUs are sent directly over layer 2 - whereas TCP (or other transport) are sent over Layer3.
This means we have a potential fate sharing issue where IIHs (which continue to be sent over Layer 2 in your proposal (as I understand it) may be successfully exchanged but the transport session set up to send LSPs/SNPs may fail. This is the exact issue RFC 6213 has addressed.

What should happen if IIHs are being successfully exchanged, the neighbors both indicate support for the extension, but the transport session fails to come up or goes down?
Would you follow similar behavior to RFC 6213 i.e., bring the adjacency down? If so, how would you determine when you can bring the adjacency up?

I appreciate you have indicated this as TBD in Section 7.6 of the draft, but you also say in Section 7.5 that adjacency should NOT go down when transport session fails - which implies we can have an ongoing condition where an adjacency is up but 
flooding cannot be done-  which is a pathological condition.

2)Your statements regarding existing flooding limitations of IS-IS are rather dated. Many years ago implementations varied from the base specification by allowing much faster flooding and contiguous flooding bursts on an interface in support of fast convergence. There are existing and successful deployments of an instance with thousands of neighbors and thousands of nodes in a network and sub-second convergence is supported. So the statement that the existing protocol per interface flooding is a blocking factor in highly meshed topologies is not (IMO) accurate. The more accurate characterization of the flooding problem in dense topologies is the amount of redundant flooding (i.e., a node may receive many copies of a new LSP) - which your proposal does nothing to address (I understand it was not intended to address this problem which you discuss in a different draft).

My point here is that there are existing implementations which would get no benefit from your proposal. It might be argued that someone writing a new implementation may find it simpler to make use of a transport mechanism like TCP - but I do not think there is compelling data that demonstrates that the scalability of an implementation using your proposal is better than that of many existing implementations.

This then suggests that for existing implementations the main motivation to support your proposal is to help other implementations which have not optimized their existing implementations. :-)
Comments? 

   Les

> -----Original Message-----
> From: Lsr <[email-masked]> On Behalf Of Henk Smit
> Sent: Monday, November 05, 2018 8:22 PM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: [Lsr] IS-IS over TCP
> 
> 
> Thanks, Tony.
>

---

From: [email-masked]
Date: Wed, 7 Nov 2018 11:38:13 +0700

Les,

> On Nov 7, 2018, at 11:09 AM, Les Ginsberg (ginsberg) <[email-masked]> wrote:
> 

If there are undocumented implementation enhancements, then other implementations cannot be presumed to have implemented them.  Other implementors are fully within their rights to propose other solutions to the same issues that the undocumented enhancements address.

My recommendation would be to document these enhancements.  However, if you choose not to, I think that the WG is fully within its rights to pursue other solutions.

Regards,
Tony

p.s. Nothing is new again.  This is analogous to the situation with HSRP/VRRP.  Have we learned from that?

---

From: Tony Przygienda <[email-masked]>
Date: Wed, 7 Nov 2018 11:50:39 +0700

>
> 2)Your statements regarding existing flooding limitations of IS-IS are
> rather dated. Many years ago implementations varied from the base
> specification by allowing much faster flooding and contiguous flooding
> bursts on an interface in support of fast convergence. There are existing
> and successful deployments of an instance with thousands of neighbors and
> thousands of nodes in a network and sub-second convergence is supported. So
> the statement that the existing protocol per interface flooding is a
> blocking factor in highly meshed topologies is not (IMO) accurate. The more
> accurate characterization of the flooding problem in dense topologies is
> the amount of redundant flooding (i.e., a node may receive many copies of a
> new LSP) - which your proposal does nothing to address (I understand it was
> not intended to address this problem which you discuss in a different
> draft).
>
>
unsurprisingly +1 here ...

Further, if you start to go that path (i.e. max. flooding transport speed
which contrary to Les I think does have value but not necessarily for ISIS)
and you _really_ care about max. flooding speed you end up most likely in
_go over UDP_ route as people like bitttorent or QUIC guys went. There is a
good reason behind that ;-) Moreover, if you form adjacencies that way
you'll find that MT/MI problems are fairly trivial to resolve. Problem
being though taht the 'well-known port', 'well-known-protocol',
'well-known-ethertype' allowing for policing of the protocol vanishes and
you have to be smarter ...

two old hands' gave you two cents ;-)

--- tony

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 7 Nov 2018 09:39:11 +0100

Hi Les,

> There are existing and successful deployments of an instance with
thousands of
> neighbors and thousands of nodes in a network and sub-second convergence
is supported.

While I completely agree with your above observation (and as a matter of
fact first person to tell me that was actually Henk around year 2000) may I
ask what problem are we solving with vast majority of the proposals in
space of "IGP scaling" for data center deployments currently on the table ?
Are we perfecting the perfect ?

It sounds that what is really needed is a deployment guide (perhaps in the
form of informational RFC) documenting that use of ISIS in 1000(s) node DC
cluster is fine provided features X, Y, Z are enabled.

Otherwise folks out there in the field no matter what is done here will
keep using BGP for huge 50 node DC clusters because RFC7938 says so.

Cheers,
Robert.

On Wed, Nov 7, 2018 at 5:09 AM Les Ginsberg (ginsberg) <[email-masked]>
wrote:

> This means we have a potential fate sharing issue where IIHs (which
> continue to be sent over Layer 2 in your proposal (as I understand it) may
> be successfully exchanged but the transport session set up to send
> LSPs/SNPs may fail. This is the exact issue RFC 6213 has addressed.
>
> What should happen if IIHs are being successfully exchanged, the neighbors
> both indicate support for the extension, but the transport session fails to
> come up or goes down?
> Would you follow similar behavior to RFC 6213 i.e., bring the adjacency
> down? If so, how would you determine when you can bring the adjacency up?
>
> I appreciate you have indicated this as TBD in Section 7.6 of the draft,
> but you also say in Section 7.5 that adjacency should NOT go down when
> transport session fails - which implies we can have an ongoing condition
> where an adjacency is up but
> My point here is that there are existing implementations which would get
> no benefit from your proposal. It might be argued that someone writing a
> new implementation may find it simpler to make use of a transport mechanism
> like TCP - but I do not think there is compelling data that demonstrates
> that the scalability of an implementation using your proposal is better
> than that of many existing implementations.
>
> This then suggests that for existing implementations the main motivation
> to support your proposal is to help other implementations which have not
> optimized their existing implementations. :-)
> Comments?
>
>    Les
>
>

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Wed, 7 Nov 2018 08:54:44 +0000

Robert –

The comment being discussed here is in regards to the potential benefits of altering IS-IS to use a different way of flooding over an interface.
Whether we agree/disagree on this does not have any bearing on the high amount of redundant flooding which occurs in a highly meshed network – which I think all of us agree is a problem worth addressing.

The latter is addressed in a number of drafts – including Henk/Gunter’s https://datatracker.ietf.org/doc/draft-hsmit-lsr-isis-dnfm/ .

But this thread is about https://datatracker.ietf.org/doc/draft-hsmit-lsr-isis-flooding-over-tcp/ .

Please do not confuse the two.

   Les

From: Robert Raszuk <[email-masked]>
Sent: Wednesday, November 07, 2018 12:39 AM
To: Les Ginsberg (ginsberg) <[email-masked]>
Cc: [email-masked]; Tony Li <[email-masked]>; [email-masked]
Subject: Re: [Lsr] IS-IS over TCP

Hi Les,

> There are existing and successful deployments of an instance with thousands of
> neighbors and thousands of nodes in a network and sub-second convergence is supported.

While I completely agree with your above observation (and as a matter of fact first person to tell me that was actually Henk around year 2000) may I ask what problem are we solving with vast majority of the proposals in space of "IGP scaling" for data center deployments currently on the table ? Are we perfecting the perfect ?

It sounds that what is really needed is a deployment guide (perhaps in the form of informational RFC) documenting that use of ISIS in 1000(s) node DC cluster is fine provided features X, Y, Z are enabled.

Otherwise folks out there in the field no matter what is done here will keep using BGP for huge 50 node DC clusters because RFC7938 says so.

Cheers,
Robert.

On Wed, Nov 7, 2018 at 5:09 AM Les Ginsberg (ginsberg) <[email-masked]<mailto:[email-masked]>> wrote:
Henk/Gunter -

Couple of points.

   Les

> From: Lsr <[email-masked]<mailto:[email-masked]>> On Behalf Of Henk Smit
> To: [email-masked]<mailto:[email-masked]>
> Cc: [email-masked]<mailto:[email-masked]>
> [email-masked]<mailto:[email-masked]> schreef op 2018-11-06 04:51:
> > [email-masked]<mailto:[email-masked]>
> [email-masked]<mailto:[email-masked]>
_______________________________________________
Lsr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/lsr

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 7 Nov 2018 10:09:37 +0100

Hi Les,

I am very well aware about different scopes of both drafts. Actually I read
them all.

But just wanted to make sure your point about ISIS working fine today with
1000s node networks is really a prove that flooding optimisation is still
*practically* needed/required.

Btw ... Henk has a second proposal - a companion document to flooding over
TCP on flooding reduction too. Perhaps you missed it :)

https://datatracker.ietf.org/doc/draft-hsmit-lsr-isis-dnfm/

Thx,
R.

On Wed, Nov 7, 2018 at 9:54 AM Les Ginsberg (ginsberg) <[email-masked]>
wrote:

> Robert –
>
>
>
> The comment being discussed here is in regards to the potential benefits
> of altering IS-IS to use a different way of flooding over an interface.
>
> Whether we agree/disagree on this does not have any bearing on the high
> amount of redundant flooding which occurs in a highly meshed network –
> which I think all of us agree is a problem worth addressing.
>
>
>
> The latter is addressed in a number of drafts – including Henk/Gunter’s
> https://datatracker.ietf.org/doc/draft-hsmit-lsr-isis-dnfm/ .
>
>
>
> But this thread is about
> https://datatracker.ietf.org/doc/draft-hsmit-lsr-isis-flooding-over-tcp/ .
>
>
>
> *From:* Robert Raszuk <[email-masked]>
> *Sent:* Wednesday, November 07, 2018 12:39 AM
> *To:* Les Ginsberg (ginsberg) <[email-masked]>
> *Cc:* [email-masked]; Tony Li <[email-masked]>; [email-masked]
> *Subject:* Re: [Lsr] IS-IS over TCP
>
>
>
> Hi Les,
>
>
>

---

From: Henk Smit <[email-masked]>
Date: Wed, 07 Nov 2018 14:26:04 +0100

Hi Les,
Comments inline.

Les Ginsberg (ginsberg) schreef op 2018-11-07 05:09:

> continue to be sent over Layer 2 in your proposal (as I understand it)
> may be successfully exchanged but the transport session set up to send

The problem that RFC6213 tries to solve is a case where one of the
neighbors is thinking that the other does not support BFD. And thus
the lack of BFD is not used as an indication that something is wrong.
Right ?

When using TCP, both neighbors are aware that they want/need to use TCP.
Because both advertise the capability in a TVL in the IIH.
So this problem won't occur. The solution of RFC 6213 is to include
a new BFD-TLV in the IIHs. Well, our proposal also includes a TLV in
the IIH. Same solution.

Another thing that might be noteworthy is that we propose to use
ISIS-Flooding-over-TCP only on p2p interfaces. Because we believe
that the majority of large IS-IS networks (in data-centers at least)
will configure all their ethernet-interfaces to be p2p.
The problem-description in section 2 in RFC6213 seems to indicate
a situation with 3 routers (A, B and C) connected via one interface.
Correct ? So that's a multi-point interface. If we would want to do
Flooding-over-TCP on multi-point interfaces, we will need a much
more complex algorithm (involving the DIS, and potentially a
backup DIS too). Not worth it, imho. The benefits of this proposal
is that a) it's simple, and b) it's using existing technology (TCP).

> What should happen if IIHs are being successfully exchanged, the
> neighbors both indicate support for the extension, but the transport
> session fails to come up or goes down?
> Would you follow similar behavior to RFC 6213 i.e., bring the
> adjacency down? If so, how would you determine when you can bring the
> adjacency up?

That is a good question.

> I appreciate you have indicated this as TBD in Section 7.6 of the
> draft, but you also say in Section 7.5 that adjacency should NOT go
> down when transport session fails - which implies we can have an
> ongoing condition where an adjacency is up but

My gut feeling says we should not tear down the adjacency.

What if two routers can exchange IIHs and do proper flooding of
LSPs. But they can not exchange IP packets ? This could happen.
IS-IS does not have a way to deal with this. IS-IS would keep
advertising the link to the rest of the network. And blackhole
traffic. But people have never complained about this risk before.
Why ? My guess is that implementations are good enough that we
know ip-forwarding almost works. It is my opinion that we can
be just as sure that TCP between two directly connected routers
will work just as good. If not, it's probably a bug. Just as likely
that ip-forwarding won't work between the same two routers.

The reason I propose to not tear down the adjacency is that it
makes other things much more complex. The state of the TCP connection
itself would introduce new state that makes it harder to do things
like non-stop-forwarding, graceful restart, process restarts,
fail-over to a hot-standby 2nd control-plane, etc.

> bursts on an interface in support of fast convergence. There are
> existing and successful deployments of an instance with thousands of
> neighbors and thousands of nodes in a network and sub-second
> convergence is supported. So the statement that the existing protocol
> per interface flooding is a blocking factor in highly meshed
> topologies is not (IMO) accurate. The more accurate characterization
> of the flooding problem in dense topologies is the amount of redundant
> flooding (i.e., a node may receive many copies of a new LSP) - which
> your proposal does nothing to address (I understand it was not
> intended to address this problem which you discuss in a different
> draft).

I worked on IS-IS in the nineties. Yes, my knowledge is outdated.
However, I've not seen any RFC or draft or documents that indicate
how to do better flooding. That means implementors have to reinvent
the wheel. Packet-pacing, high-bandwidth, back-pressure,  etc will
steer implementations towards proprietary solutions that will resemble
things that TCP does. If so, why not use TCP ? Or another existing
transport protocol that does even better ?

We're trying to solve flooding issues. In very dense networks. And
in networks with many routers. The requirements document that I read
earlier this year mentions 10k routers. Tony Li suggested last month
to me that flooding algorithms should be able to deal with situations
where single routers have 1k adjacencies. If we want to improve the
protocol, I think we should also improve the situation where we need
to flood 10k LSPs over a single adjacency. That's the problem we try
to solve here.

> My point here is that there are existing implementations which would
> get no benefit from your proposal. It might be argued that someone
> writing a new implementation may find it simpler to make use of a
> transport mechanism like TCP - but I do not think there is compelling
> data that demonstrates that the scalability of an implementation using
> your proposal is better than that of many existing implementations.

When I worked at cisco in the nineties, that IS-IS implementation
could deal with 250 routers in a double full mesh. That's 500
adjacencies per router. Those routers were running 100MHz and 200MHz
mips cpus. Some were even cisco 7000s with 68040 cpus. That worked.
With my outdated knowledge, I can not understand who a datacenter
fabric where routers have 64 or 128 adjacencies could be a problem.
Not with a proper implementation. But we're trying to fix that too.

> This then suggests that for existing implementations the main
> motivation to support your proposal is to help other implementations
> which have not optimized their existing implementations. :-)
> Comments?

Proper IS-IS implementations should split up in 3 threads at least.
One thread for maintaining adjacencies, one for doing flooding and
one for doing SPFs (and route-installs). That way an SPF doesn't
hold up flooding. And heavy flooding or long SPFs don't break
adjacencies. How many implementations in the field do that ?
Heck, I've even seen today's implementations that do not split
off adjacency-maintenance as a separate process or thread.

Anyway, the question is: if you want to have 10k LSPs in your
flooding domain, do we depend on custom improvements, or do
we want something that's documented ?

One of the things that inspired me to do this proposal was LSVR.
LSVR uses BGP-LS to transport LSPs. Why ? The word on the street
is that "BGP scales so much better". Why does BGP scale so good ?
Imho there is one main reason: BGP uses TCP for transport.
Now I don't like LSVR (and I don't like BGP-LS). Because LSVR
seems to re-invent the wheel, with no real improvements over
IS-IS, except for the fact that it uses BGP which uses TCP.
If that's the main benefit of LSVR, then why not just use
TCP and be done with it ?

BTW, there is another reason for our proposal. With the incoming
drafts about flooding-topology-reduction, there is a new problem.
All these proposals have situations where non-flooding adjacencies
suddenly change to flooding adjacencies. When that happens, the
LSDBs need to be synchronized again. To do that, all of them
propose "just send a CSNP and be done with it". Well, the more
LSPs, the more CNSPs that need to be sent. With 10k LSPs that's
110 CSNPs. CSNPs are not reliable. This re-synchronization happens
when there is churn in the IGP. Are we sure CSNPs aren't dropped
somewhere ? Can we start sending LSPs because we know the neighbor
has sent all its CSNPs yet ? With reliable transport for LSPs and
SNPs these worst-case scenarios will improve.

Apologies for the long text.
I hope it explains our goals and proposal a bit more.

henk.

---

From: Henk Smit <[email-masked]>
Date: Wed, 07 Nov 2018 15:06:45 +0100

Hi Jeff,

> I'm ambivalent of the transport, but agree that TCP shouldn't be the 

I picked TCP because every router has a working TCP implementation.
And TCP is good enough for BGP. And thus also considered good enough
for LSVR. If that's the case, I'd assume it is good enough for IS-IS
as well.

It's easy to adjust our draft so that new transports can be introduced
over time. We can do TCP now. Add Quic later. And add other, new, better
transports later, when they become available.

I don't know much about Quic. But it seems the protocol and details are
not 100% stable yet. Maybe soon. So maybe we can do TCP now, and Quic 
later ?
Also, Quic might be easy to implement for router OSes that run on top
of Unix. But for OSes that use QNS, vxWorks, or something proprietary,
Quic might be more work. (It's up to others to decide if that's 
important
or not. I have no opinion on this matter myself).

>   packet loss or slow receivers.

If a receiver is slow, that's the same situation as when IS-IS on the
neighbor is slow. Retransmissions happen. Retransmissions with 10589
flooding are fixed time (5 seconds). (I guess some modern 
implementations
do something smarter). So convergence would have been impacted with
native flooding too.

Note that our proposal only does TCP over directly connected routers.
I'm sorry to say I have little experience with behaviour of BGP in
real networks. Where did you observe these backpressure issues ? EBGP
or iBGP ? I expect to see more problems with iBGP, because iBGP goes
over multiple hops, which can cause all kinds of issues. EBGP is mostly
over directly connected interfaces. I expect TCP to behave much nicer
There. TCP behaviour of IS-IS flooding would resemble eBGP more than 
iBGP.
At least, that is my expectation.

Note. If you would do flooding over a tunnel, flooding over TCP might
be beneficial too. Because of tunnel overhead (e.g. GRE-headers) tunnels
usually have a smaller MTU. Therefor all max-sized LSPs will need to be
fragmented when sent over the tunnel. This is also (especially) true for
CSNPs. For packet-based tunneling-protocols, that means 2 packets for
each max-sized LSP or CSNP. When using TCP, the LSPs get spread out over
multiple segments, which should make reassembly a bit easier/cheaper.

>   sensitive things.  This includes something as long as 3 second BGP 

When you do flooding-over-tcp, then you don't need to send PSNPs (acks) 
or
do retransmission of LSPs. So you don't need timers for those. Things 
become
less time-sensitive (at the cost of potentially slower flooding).

>   "stuck" flooding due to backpressure or other things, some of these 

Again, it is my expectation that in case of problems with TCP, that same 
situation
would have been worse with native ISIS flooding.

Also note that in BGP, every update packet over every peering has 
significance.
If one gets delayed, it slow down overall convergence. In ISIS flooding, 
a router
will receive multiple copies of the same LSP. So if one TCP-connection 
is slow,
the router might still receive the same LSPs over other paths. And the 
impact
on overall convergence is likely to be less.

Of course, this implies that routers flood over more than 1 or 2 
interfaces. If
we do one of the flooding-reduction proposals, I hope we'll end up with 
a situation
where we have 3 or 4 redundant flooding topologies, so that routers will 
still
receive LSPs quickly over other topologies when the primary topology has 
problems.

> given a look at based on the comments above.

One other benefit of doing flooding over TCP is that part of the 
flooding
administration is now outsourced to TCP. And TCP usually runs in another 
thread
of process, inside or outside the kernel. This means that we'll 
automatically
get a light form of multi-threading. Less work for the IS-IS 
process/threads.
Quic runs in user-space. I don't know if that means it is a library, and 
functions
run in the user's thread/process. Or whether Quic is a separate 
process/thread.
If Quic runs in IS-IS's thread, it means we lose a cheap form 
performance
improvements because of multi-threading.

henk.

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Wed, 7 Nov 2018 16:06:18 +0000

Henk -

Thanx for the thoughtful response.
I'll do my best to respond in kind.
Inline.

> From: Henk Smit <[email-masked]>
> Sent: Wednesday, November 07, 2018 5:26 AM
> Cc: [email-masked]; [email-masked]
[Les:] This is not correct.
The key paragraph is in https://tools.ietf.org/html/rfc6213#section-2 

" The problem with this solution is that it assumes that the
   transmission and receipt of IS-IS Hellos (IIHs) shares fate with
   forwarded data packets.  This is not a fair assumption to make given
   that the primary use of BFD is to protect IPv4 (and IPv6) forwarding,
   and IS-IS does not utilize IPv4 or IPv6 for sending or receiving its
   hellos."

We have seen cases where IPv4/IPv6 data packet delivery has been compromised - but IS-IS PDU delivery was unaffected. This led to the following behavior:

1)IS-IS exchanges hellos and bring adjacency up. Routes using the link are installed
2)BFD session is started and comes up
3)After a time some problem occurs which only impact IP traffic - BFD session goes down.
4)IS-IS adjacency is brought down due to BFD session down, but IS-IS continues to send hellos. If they are successfully exchanged then IS-IS adjacency is almost immediately restored and we resume installing IP routes using the link even though BFD session never comes up. Data traffic gets dropped.

The extensions in RFC 6213 allow IS-IS to know when both sides support BFD, which means the sequence changes to:

1)IS-IS exchanges hellos - but adjacency remains in INIT state.
2)BFD session is initiated - IS-IS adjacency remains in INIT state until BFD session comes up.

Thus IS-IS never installs routes using the link unless we know IP traffic can be successfully delivered.
We can then use BFD both as a requirement to bring the adjacency up AND as fast failure detection.

> Correct ? 

[Les:] RFC 6213 is applicable to Pt-Pt interfaces - in fact it is easier to utilize on Pt-Pt as we do not have the N-squared issue. It is unfortunate that the example in the RFC uses a LAN.

> So that's a multi-point interface. If we would want to do
> IS-IS does not have a way to deal with this.

[Les:] RFC 6213 was written precisely to address this case - and works very well.

> IS-IS would keep

[Les:] Actually they have. :-) That's why we wrote RFC 6213 - because the problem has been seen in the field.

> fail-over to a hot-standby 2nd control-plane, etc.

[Les:] I appreciate your concerns - and it is true that "most of the time" if we cannot receive IP traffic we likely cannot receive IS-IS PDUs . But actual field experience has shown this is not always the case - and when it is not the lack of recognition of the failed state has major impacts.

I don't think you can ignore this case just because it will happen infrequently.

> to flood 10k LSPs over a single adjacency. That's the problem we try

[Les:] I am not suggesting that this aspect of scaling can be ignored. But I am saying implementations have successfully addressed this with "smarter implementations" and done so w/o requiring protocol extensions.

> adjacencies. How many implementations in the field do that ?

[Les:] You are correct - and the better implementations do exactly what you describe. They had to because problems encountered in real deployments required it.

> we want something that's documented ?
[Les:] I think the details of the "smart implementations" have not been formally documented because they did not require protocol extensions (i.e., they are interoperable w implementations which do not have equal smarts) and they are largely seen as the value add resulting from significant development/test efforts. Why would I want to give that knowledge away for free when I might use it to gain a competitive advantage?

Perhaps it is time to revisit that and consider writing some sort of BCP document - but that is a separate discussion.

I am simply stating what I know to be true.

> SNPs these worst-case scenarios will improve.
[Les:] I do not advocate "just send an CSNP and be done with it" for the very reasons you mention.
You will see that https://tools.ietf.org/html/draft-li-lsr-dynamic-flooding-01#section-6.7.2 specifies that normal LSPDB synchronization occurs on adjacency bringup.

> I hope it explains our goals and proposal a bit more.

[Les:] No apology necessary - it is a good discussion.
Hope my replies have been helpful.

   Les

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 7 Nov 2018 14:56:18 -0500

Henk,

> I picked TCP because every router has a working TCP implementation.

Having just done a history lesson on BGP for NLNOG, it's sort of fun to see
this. :-)

> do something smarter). So convergence would have been impacted with

I guess my question to those who live in IGP land is how often is this a
problem?  In the case of an IGP, the backpressure means you have databases
that are out of sync and end up with bad forwarding.  

> or iBGP ?

Both iBGP and eBGP.  The two general issues are slow receivers (scale) or
responses to dropped packets.

The general experiment I recommend to people trying to do this sort of thing
is take your TCP stream of choice, pace it according to your transmission
needs, and then drop 5-30% of the packets.  Observe what happens.

TCP recovers fine.  But the hiccups can do bad things when timeliness is
expected.  For example, 3 second hold times for aggressive BGP peering may
time out.

I guess I'd restate my concern as "for this application, ensure that you're
okay with the results of stalled trasnmission".  Effectively, see the answer
to the question I asked above about native behaviors.

> If one gets delayed, it slow down overall convergence.

This is true.  It's usually survivable due to hop-by-hop forwarding in BGP's
case, but it can still do bad things.

>  In ISIS
> flooding, a router
> will receive multiple copies of the same LSP. So if one
> TCP-connection is slow,
> the router might still receive the same LSPs over other paths. And
> the impact

This, I think, is a better point addressing my concerns.  Thanks.

> we do one of the flooding-reduction proposals, I hope we'll end up
> with a situation
> where we have 3 or 4 redundant flooding topologies, so that routers
> will still
> receive LSPs quickly over other topologies when the primary topology
> has problems.

:-)

-- Jeff

---

From: Henk Smit <[email-masked]>
Date: Wed, 07 Nov 2018 22:35:23 +0100

> that are out of sync and end up with bad forwarding.

As discussed below, if you have multiple flooding paths, and not all of
them are congested or throttled, when at least one copy of the LSP makes
it across, convergence will be fast.

> responses to dropped packets.

Slow receivers are a problem for native flooding too.
Although I suspect you mean: after congestion problems, and the 
situation
improves, native flooding will recover quickly, while TCP might 
intentionally
keep things slow for a longer period of time. Correct ?
Yes, that is an issue.

> needs, and then drop 5-30% of the packets.  Observe what happens.

Don't we have DSCP for this ?
And remember, the TCP connection will (almost) always be between two
directly connected endpoints.

> expected.  For example, 3 second hold times for aggressive BGP peering 

Our proposal is to do only flooding over TCP.
Adjacency management is still done based on native IIHs (and BFD).
Even if TCP stalls the flooding, the adjacency should stay up.
With flooding over multiple paths, it should not be a fatal event.

> to the question I asked above about native behaviors.

[Flooding happens over multiple paths. As long as one path is quick, 
convergence
  will be quick too].

> This, I think, is a better point addressing my concerns.  Thanks.

I expect there will be more issues that need to be addressed.
E.g. an old rule of thumb is: don't generate a routing update (packet) 
unless
you are pretty sure you can send it right away, and the receiver can 
receive it.
Otherwise a lot of the actual communication might be stale information.

I'm not sure if people find this rule of thumb still relevant these 
days.
(I know people who do not). With an abundance of cpu-cores, memory and 
bandwidth,
it seems many problems of the past are not visible anymore. Unless you 
start
pushing beyond what most people do. But if you do care, it is advisable 
to keep
your TCP window-sizes small. Maybe at the default 16KB, maybe even 8KB 
or 4KB.
With a window-size of 4KB you might be able to still send a dozen 
average-size LSPs,
and those might get stuck/stale in TCP. But I think that's a good 
trade-off to get
syncing of large LSDBs. As long as you don't set the window-size to 64KB 
or larger.
And maybe even then, stale LSPs might be less of a problem than 
old-timers think.

henk.

---

From: Henk Smit <[email-masked]>
Date: Wed, 07 Nov 2018 23:21:31 +0100

Les Ginsberg (ginsberg) wrote 2018-11-07 17:06:

> We have seen cases where IPv4/IPv6 data packet delivery has been
> compromised - but IS-IS PDU delivery was unaffected. This led to the
> following behavior:
> 
> 4)IS-IS adjacency is brought down due to BFD session down, but IS-IS
> continues to send hellos. If they are successfully exchanged then
> IS-IS adjacency is almost immediately restored and we resume
> installing IP routes using the link even though BFD session never
> comes up. Data traffic gets dropped.

Thanks for the explanation.
But that is kinda what I wanted to say.
BFD is failing (because the IP-path is failing), and IS-IS doesn't
realize this, because it thinks that BFD isn't being used (because
"BFD session never comes up".

> The extensions in RFC 6213 allow IS-IS to know when both sides support
> BFD, which means the sequence changes to:
> 

OK. What do you suggest we do to fix this failure case if we do
flooding over TCP ?

- We could make BFD mandatory when flooding over TCP ? If the IP-path
is broken, TCP will fail, but BFD will also fail ?

- We bring the adjacency to UP state, but we don't include it in our own
LSP immediately. Only after the TCP session has been established, we
advertise the new adjacency in our LSP. Would that be enough ? It would
stop routes from being calculated over the new adjacency.
Maybe wait until the TCP connection has been set up, and a pair of IIHs
has been exchanged over it ? (So authentication and other stuff can
be verified for the TCP session).
Or maybe even wait until IIHs have been sent, and then full sets of 
CSNPs
are exchanged in both directions ?

That last suggestion starts approaching the way OSPF does this. If I 
recall
correctly, OSPF will only include adjacencies in its type-1 LSA after 
DDs
have been exchanged, and the full LDSB has been synchronized. Would you
want IS-IS to do the same ?

- If the TCP session breaks, do you want to stop including the adjacency
in the LSP ? This will make things like NSF, process restart and control
plane failover much harder.

> [Les:] RFC 6213 was written precisely to address this case - and works

The fact that BFD is working does not mean it is 100% sure that aal
IP traffic will work. Failures might depend on protocol number,
portnumbers, packetsize, etc. I agree that it is likely that if BFD
works, all of IP will work. But it's no guarantee.
Likewise we have to decide how paranoia we want to be that if IIHs
are exchanged, how sure are we that TCP can exchange LSPs as well ?

Maybe a good compromise would be:
1) don't advertise the adjacency in your LSPs until the TCP flooding
connection has been established. (And maybe IIHs/CSNPs are exchanged).
2) after connection is fully up (IIHs and flooding works), use longer
time-outs to determine whether TCP is still working.
3) when the other side closes a TCP connection (by FIN or RST), don't
stop advertising the adjacency in your LSP immediately. In stead,
for the next 10 seconds or so, try to re-establish the TCP connection
first. If re-establishment doesn't work, then the router can stop
including the adjacency in its LSP.

This would prevent routers advertising new adjacencies that have a 
problem
with TCP. But if it works, and suddenly stops working, convergence is
slower (10 seconds or so). But the protocol has the ability to 
re-establish
the TCP session, to make it more flexible.

Would that be acceptable ?

> the problem has been seen in the field.

I was talking about a router forwarding traffic.
BFD between router A and router B could work, but B (for some reason)
doesn't forward traffic. That was my extreme example to show that BFD
doesn't protect all weird failure scenarios possible. The question is:
how paranoia does a protocol want to be ?

> [Les:] I appreciate your concerns - and it is true that "most of the
> time" if we cannot receive IP traffic we likely cannot receive IS-IS
> PDUs . But actual field experience has shown this is not always the
> case - and when it is not the lack of recognition of the failed state
> has major impacts.
> 

We got BFD to protect against that.
But if I understand correctly, you propose that if the TCP session 
doesn't
come up, or get torn down later, we also bring down the adjacency, even
when IIHs are still exchanged. That's reasonable, of course. Even if it 
makes
things slightly more complicated.

> [Les:] I am not suggesting that this aspect of scaling can be ignored.
> But I am saying implementations have successfully addressed this with
> "smarter implementations" and done so w/o requiring protocol
> extensions.

I personally agree with this position.
Vendors should be able to distinguish themselves from the competition.

But people working for operators have a different view.
When this draft got published, we got private email from an operator
addressing a number of issues (mainly when using unnumbered interfaces).
His point was: he wanted to make sure that all corner-cases are handled
identically by all vendors. Or at least in an inter-operable way.
Therefor he wanted every little detail of implementation explicitly
mentioned. I guess seeing your backbone go down because of 
vendor-specific
enhancements (or lack thereof) will give you a different view on the 
matter.

So I think that if vendors do things that are noticable on the wire, 
even
if they should be compatible with standard behaviour, those things 
should
be documented at least. Therefor I thought: "if all vendors are gonna
implement their own pacing and their own reliability and their own
high throughput, why not all use the same well-known transport 
protocol".
Hence our proposal.

> [Les:] I think the details of the "smart implementations" have not
> been formally documented because they did not require protocol
> extensions (i.e., they are interoperable w implementations which do
> not have equal smarts) and they are largely seen as the value add
> resulting from significant development/test efforts. Why would I want
> to give that knowledge away for free when I might use it to gain a
> competitive advantage?

Router vendors do not.
Operators probably do want everything written down.
For themselves and to ensure inter-operability.

> Perhaps it is time to revisit that and consider writing some sort of
> BCP document - but that is a separate discussion.

If the people who want this knowledge shared, can find someone who can
has that knowledge, and who wants to write it down, they should go 
ahead. :)

The thing that surprises me a little that a lot of these protocol
enhancements (or even brand new protocols) would not be necessary
if all implementations were perfect. It seems that in some cases we
try to improve networks not by improving equipment, but by trying
just a slightly different way to do things.

Again, if people think that LSVR is a good idea, then how can they
think that ISIS flooding over TCP is not a good idea ? This is
the base idea for our proposal. A quick look at the LSVR draft
show people from Cisco, Nokia (and Arrcus). (I'm not sure what
Juniper or Arista or other vendors think about using BGP-LS).

> You will see that
> https://tools.ietf.org/html/draft-li-lsr-dynamic-flooding-01#section-6.7.2
> specifies that normal LSPDB synchronization occurs on adjacency
> bringup.

That test mentioned enabling (and thus also disabling) flooding over 
specific
adjacencies. When you (re-)enable flooding over an adjacency, you need
to do the synchronization over again. So you need to exchange CSNPs and
transmit any missing LSPs. That's what I meant by writing "and be done 
with it".
It's not a complex idea. But when CSNPs get dropped, synchronization 
becomes
more expensive (because you transmit LSPs which might not have been 
necessary).
I hope using TCP will make synchronization in this case more efficient.

henk.

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 7 Nov 2018 23:41:23 +0100

Hi Henk,

> - We could make BFD mandatory when flooding over TCP ?

I don't think this would be a good idea.

See especially for p2p fiber links loss of light is a much faster and
better trigger/indicator that your link or adjacent router is down.

Using BFD there which in most cases operates in a distributed mode on the
line cards and tells you nothing about LC to RP path, or RP liveness IMO
does not make much sense in such designs.

Of course when your "link" is still p2p but is an emulated one over someone
else IP network this is different story, but I don't think this is most
common deployment model in data center's fabric.

Thx,
R.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 8 Nov 2018 01:36:19 +0100

All,

> Juniper or Arista or other vendors think about using BGP-LS).

I would like to make one additional observation here ...

We are experiencing BGP-LS crusade (completely outside of LSVR) as there is
some demand to send data carried by IGP (incl SR, TE and now even BFD
extensions) to remote controllers over TCP.

I am pretty sure that BGP-LS would have never started if we would have had
an ability to send LSDB content over TCP day one. /* Let's put controller
to controller NNI across ASes aside for a moment. */

With that to me ability to progress with TCP transport extension for ISIS
(and OSPF) is actually more important then work on flooding reduction. I
know it may not be a popular statement, but that is based on both looking
at the operational side as well as BGP protocol side.

Thx,
R.

---

From: Uma Chunduri <[email-masked]>
Date: Thu, 8 Nov 2018 01:54:28 +0000

>BTW, there is another reason for our proposal. With the incoming drafts about flooding-topology-reduction, there is a new problem.
	>All these proposals have situations where non-flooding adjacencies suddenly change to flooding adjacencies. When that happens, the LSDBs need to be synchronized again. To do that, all of them propose "just send a CSNP and be done with it". Well, the more LSPs, the more CNSPs that need to be 	>sent. With 10k LSPs that's
	>110 CSNPs. CSNPs are not reliable. This re-synchronization happens when there is churn in the IGP. Are we sure CSNPs aren't dropped somewhere ? 

These do get dropped and with default configurations of lot of commercial products, where no periodic CSNP advertisements on P2P links, caused lot of grief in even in regular SP networks (especially in some HA scenarios).
It's obvious and one has to enable recommend periodic CSNPs for any proposal relevant here (despite 110 CSNPs, yes inefficient but saves you big time in corner cases). This is kind of most needed insurance...

--
Uma C.

---

From: Uma Chunduri <[email-masked]>
Date: Thu, 8 Nov 2018 01:54:52 +0000

Fully agree with below observation.

There was lot of discussion around this when BGP-LS work started on this topic.

The key advantage BGP-LS brings in IGP agnostic transport from any one IGP node (possibly RR) to controller.

This was the advantage over “static IGP adjacency/passive peering” with controller (in this case controller has to run IS-IS/OSPFv2/OSPFv3 as opposed to single protocol BGP ==> remember early days ODL has only BGP).

But w.r.t real time updates of LSDB to controller nothing can beat passive peering (but this has proximity issue, as controller can be remote). If passive peering with controller can be done on TCP that would have been awesome (can avoid 2 conversions and BGP processing delay of LSDB eventually for PCE).

I would also note there are commercial implementations which do innovative passive peering to controller other than TCP (there are no drafts for the same); for folks who care more real time LSDB updates!

Yes, this is one side advantage of the proposal being discussed here other than MSDC underlays with crazy number of nodes and massive ECMPs.

--
Uma C.

From: Lsr [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Thursday, November 08, 2018 7:36 AM
To: [email-masked]
Cc: Les Ginsberg (ginsberg) <[email-masked]>; Tony Li <[email-masked]>; [email-masked]
Subject: Re: [Lsr] IS-IS over TCP

All,

We are experiencing BGP-LS crusade (completely outside of LSVR) as there is some demand to send data carried by IGP (incl SR, TE and now even BFD extensions) to remote controllers over TCP.

I am pretty sure that BGP-LS would have never started if we would have had an ability to send LSDB content over TCP day one. /* Let's put controller to controller NNI across ASes aside for a moment. */

With that to me ability to progress with TCP transport extension for ISIS (and OSPF) is actually more important then work on flooding reduction. I know it may not be a popular statement, but that is based on both looking at the operational side as well as BGP protocol side.

Thx,
R.

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Thu, 8 Nov 2018 02:48:31 +0000

Henk -

Top posting for better readability.
 
1)Whether we should publicly document what smart implementers have done is not relevant here (though I appreciate this thread has sparked interest). If folks want to discuss this please do so in a separate thread.
The only point I am making is that there are implementations which have already solved the problems you are trying to address by using TCP and have done so w/o requiring protocol extensions. These implementations will not benefit from your proposal.

2)Your solution introduces two sub-sessions associated with an adjacency - one the traditional hello exchange over Layer 2 - the other a "TCP-like" session used to support the IS-IS Update process (LSP flooding).
Both sub-sessions MUST be in the UP state in order for an adjacency to be usable. If either of the sub-sessions is down then the adjacency isn't usable and advertisement of the neighbor MUST be withdrawn. Otherwise we risk blackholes.

3)Because the fate of the two sub-sessions is not guaranteed share fate, you have to deal with this issue conceptually similar to the way RFC 6213 has done. This is an essential part of the solution which you must define.

4)It is true BFD does not detect all possible failures - and it is possible there are still scenarios where  BFD session state might not always share fate with the corresponding address-family data packets. If you have ideas for improvements here that would certainly be of interest to the IETF community. But BFD does work quite well and is widely deployed and its use to prevent IGP adjacency UP unless BFD session is also up has definitely solved real world problems.

5)The suppression of IS-IS adjacency advertisement until completion of initial LSPDB sync ("be like OSPF") has been addressed by use of the Suppress Adjacency(SA) bit as defined in RFC 5306 (IS-IS Restart). Note SA bit applies to cold start - not restart (do not be misled by the name of the containing document).

6)CSNP reliability is a known issue. ISO 10589 addressed this issue by requiring that SRM bit be set for all LSPs in the local LSPDB on adjacency UP in addition to sending a CSNP set. As an aside, ISO 10589 did this even though at the time (circa 1990) the underlying P2P sub-networks were all reliable (think LAPB, LAPD, X.25). Belt and suspenders..
In cases where both of the neighbors already have (most of) the LSPDB, the loss of a CSNP results in unnecessary flooding of many LSPs. Avoiding this cost may be ameliorated by making initial CSNP exchange reliable (RFC 5306 includes this as part of the restart mechanism).  But we are dealing w transitory failures - and TCP is not immune to this  The fact that no IS-IS code is required to recover from this when TCP is used does not mean that retransmissions come for free.
There is no free lunch.

   Les

> Sent: Wednesday, November 07, 2018 2:22 PM

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Thu, 8 Nov 2018 02:50:22 +0000

Another limitations with IGPs is that the adjacencies are single hop. We had proposed a solution for this for OSPF but it didn’t get a lot of momentum. It would have handled the BGP-LS requirement.

https://www.ietf.org/archive/id/draft-acee-ospf-transport-instance-03.txt

From: Lsr <[email-masked]> on behalf of Robert Raszuk <[email-masked]>
Date: Wednesday, November 7, 2018 at 7:36 PM
To: "[email-masked]" <[email-masked]>
Cc: "Les Ginsberg (ginsberg)" <[email-masked]>, Tony Li <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [Lsr] IS-IS over TCP

All,

Thx,
R.

---

From: [email-masked]
Date: Thu, 8 Nov 2018 09:59:19 +0700

Les,

> The only point I am making is that there are implementations which have already solved the problems you are trying to address by using TCP and have done so w/o requiring protocol extensions. These…

You are the one who started this thread. You are the one who said that this solution is not necessary because you have your own proprietary secret solution to the problem. 

You cannot expect anyone to take that position seriously. Other people want the problem solved and references to secret solutions is simply teasing.

Tony

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Thu, 8 Nov 2018 03:33:24 +0000

Tony -

I am not "teasing".
I also don't think this is much of a secret.

I am pretty sure experienced implementers know what I am talking about.
(BTW, I include you in that group. :-) )

I also am pretty sure many of our customers know what I am talking about since it is their scale requirements that have forced us to improve our implementations.

What I am very concerned about is that we are focused on the wrong problem. We don't have a problem flooding a large number of LSPs on a single interface.
We do have a problem flooding the same LSP redundantly on many different links in a densely meshed network.

   Les 

> From: Tony Li <[email-masked]> On Behalf Of [email-masked]
> Sent: Wednesday, November 07, 2018 6:59 PM
> Cc: Henk Smit <[email-masked]>; [email-masked]
> Subject: Teasing us with secrets
> 
> 
> Les,
> 
> > 1)Whether we should publicly document what smart implementers have done
> is not relevant here (though I appreciate this thread has sparked interest). If
> folks want to discuss this please do so in a separate thread.
> > The only point I am making is that there are implementations which have
> already solved the problems you are trying to address by using TCP and have
> done so w/o requiring protocol extensions. These implementations will not
> benefit from your proposal.
> >
> 
> 
> You are the one who started this thread. You are the one who said that this
> solution is not necessary because you have your own proprietary secret solution
> to the problem.
> 
> You cannot expect anyone to take that position seriously. Other people want
> the problem solved and references to secret solutions is simply teasing.
> 
> Tony

---

From: [email-masked]
Date: Thu, 8 Nov 2018 10:40:16 +0700

Les,

> (BTW, I include you in that group. :-) )

Ongoing undocumented knowledge is wholly irrelevant when writing standards.  There’s simply nothing to reference and thus a tease.

It’s true that there is no obligation to document this knowledge, but at the same time, that knowledge cannot be used to pre-empt alternative solutions.

> What I am very concerned about is that we are focused on the wrong problem. We don't have a problem flooding a large number of LSPs on a single interface.

The fact that there is undocumented knowledge that completely contradicts established standards that is necessary for scalable operations implies that there is indeed a problem.

Tony

---

From: "Van De Velde, Gunter (Nokia - BE/Antwerp)" <[email-masked]>
Date: Mon, 12 Nov 2018 09:53:16 +0000

GV> I agree with Tony. Maybe the ISISoTCP draft is addressing the wrong problem, but the WG should either explain or should not use undocumented knowledge as argument. Undocumented is undocumented. Undocumented information is not a normative position to say "only the happy few know, and we are not explaining". That seems not a nice and friendly and appropriate position to defend. The authors of "draft-hsmit-lsr-isis-flooding-over-tcp-00"  tried to deliver an educated objective description and explanation why there is an issue with the current standard behavior, and sofar I did not receive any indication at all, what is wrong or incorrect with the documented issues in the draft-hsmit-lsr-isis-flooding-over-tcp-00 draft. 

G/

-----Original Message-----
From: Lsr <[email-masked]> On Behalf Of [email-masked]
Sent: Thursday, November 8, 2018 04:40
To: Les Ginsberg <[email-masked]>
Cc: [email-masked]; Henk Smit <[email-masked]>
Subject: Re: [Lsr] Teasing us with secrets

Les,

Tony

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Mon, 12 Nov 2018 16:47:54 +0000

Tony -

Discussion of bypassing the ISO 10589 flooding pacing timer was done in public many years ago when sub-second convergence was introduced.
Here is one public paper:

https://inl.info.ucl.ac.be/publications/achieving-sub-second-igp-convergence-.html

There are also multiple vendor specific configuration guides which discuss tuning LSP pacing for fast convergence - please consult your favorite vendor's documentation links (not my place to share such links).

No doubt others can find other public references.

I would also point out that the initial exchange of LSPDB occurs on adjacency UP - which is a "good news" event for which there are no tight time constraints. Premature advertisement of a new adjacency in IS-IS has been addressed via the SA bit in RFC 5306. So the performance of flooding per link isn’t a problem we need to solve.

   Les

> Sent: Wednesday, November 07, 2018 7:40 PM
> Subject: Re: Teasing us with secrets
> 
> 
> Les,
> 
> > I am not "teasing".
> Ongoing undocumented knowledge is wholly irrelevant when writing
> standards.  There’s simply nothing to reference and thus a tease.
> 
> It’s true that there is no obligation to document this knowledge, but at the
> same time, that knowledge cannot be used to pre-empt alternative
> solutions.
> 
> 
> > What I am very concerned about is that we are focused on the wrong
> problem. We don't have a problem flooding a large number of LSPs on a
> single interface.
> 
> The fact that there is undocumented knowledge that completely contradicts
> established standards that is necessary for scalable operations implies that
> there is indeed a problem.
> 
> Tony
>

---

From: [email-masked]
Date: Mon, 12 Nov 2018 09:13:44 -0800

Les,

> No doubt others can find other public references.

Thank you for the reference, that’s helpful.  

Are you aware of the standards status of this document vs ISO 10589?

It seems to me that unless there’s some document on the standards track that the average developer might not comply with it.

Tony

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Mon, 12 Nov 2018 17:44:27 +0000

Tony -

AFAIK no one has suggested or is planning to update/modify a standard.

This falls into a category  of standard deviations that was initially documented  in https://tools.ietf.org/html/rfc3719 .
If the WG feels this is important enough perhaps the best thing to do is update that RFC.

I would however like us to agree that the subject of this thread is inappropriate. :-)

   Les

> Sent: Monday, November 12, 2018 9:14 AM
> > Discussion of bypassing the ISO 10589 flooding pacing timer was done in
> public many years ago when sub-second convergence was introduced.
> > There are also multiple vendor specific configuration guides which discuss
> tuning LSP pacing for fast convergence - please consult your favorite vendor's
> documentation links (not my place to share such links).
> >
> It seems to me that unless there’s some document on the standards track
> that the average developer might not comply with it.
> 
> Tony

---

From: [email-masked]
Date: Mon, 12 Nov 2018 09:48:31 -0800

Les,

As you’ve said in person, following the LSP transmission and re-transmission timer requirements is one of the first things to go.

Yes, updating the RFC with all of the other learnings would be one way forward.

Tony

> On Nov 12, 2018, at 9:44 AM, Les Ginsberg (ginsberg) <[email-masked]> wrote:
> 
> Tony -
>

---

From: "Yutianpeng (Tim)" <[email-masked]>
Date: Mon, 12 Nov 2018 18:28:53 +0000

Hi,
Personally, I believe now we should have more best practices and recommendations comparing with 2004.
Also it worth once update when segment routing comes.
Tim

-----Original Message-----
From: Lsr [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: 12 November 2018 17:49
To: Les Ginsberg <[email-masked]>
Cc: [email-masked]; Henk Smit <[email-masked]>
Subject: Re: [Lsr] Teasing us with secrets

Les,

Tony

>> tuning LSP pacing for fast convergence - please consult your favorite 
>> vendor's documentation links (not my place to share such links).
>>> 
>> It seems to me that unless there’s some document on the standards 
>> track that the average developer might not comply with it.
>> 
>> Tony
>

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 12 Nov 2018 19:35:12 +0100

IETF seems to be doing great job in defining protocols and their
extensions. It doesn't do that great job in defining the dynamics of
protocols and their recommended behavior. That's called a "vendor's secret
sauce".

This thread seems to be a good example of that.

In BGP there was similar discussions on number of timers which BIPs (best
implementation practices) are well known to limited circle of folks.
Sometimes instead of such timers different safety checks are implemented
under the hood. Take BGP MRAI as example ... some implementations do it one
way some completely other and claim our MRAI is 0. But when asked "Do you
always generate bgp update immediately" -- you hear "Oh NO !"

Draft written by Paul to make it easy to implement for those from outside
the circle did not go far in spite of becoming IDR WG doc:
https://tools.ietf.org/html/draft-ietf-idr-mrai-dep-04

Here we have a proposal to replace part of that sauce with TCP and maybe
QUIC in a backwards compatible fashion to allow for more food diversity.
Yes it is a burden for existing chefs to now have to handle one more in/out
channel, but let's realize that if we keep status quo alternative solutions
to running IGPs may be winning more markets ...

Kind regards,
Robert.

On Mon, Nov 12, 2018 at 6:48 PM <[email-masked]> wrote:

>
> Les,
>
> As you’ve said in person, following the LSP transmission and
> re-transmission timer requirements is one of the first things to go.
>
> > This falls into a category  of standard deviations that was initially
> documented  in https://tools.ietf.org/html/rfc3719 .
> vendor's