Subject: [Idr] new thread regarding capabilities handling

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 27 Apr 2017 07:18:24 +0200 (CEST)

Hi,

disregard the part in my last email about asking why all bgp sessions in a 
peer group are flapped when one changes the config of that peer group. 
Let's discuss that here instead.

If I add "address-family ipv4 multicast" to my peer-group and it wasn't 
there before, all router OSes I know will just reset all the sessions. 
This is baffling.

I see two problems:

1. Why can't capabilities be renegotiated without BGP session restaart?

2. Why doesn't it require a command to flap the sessions to adapt to the 
new configuration? If I want to run-through my changed policy-map, I have 
to "clear soft in" (or out). Why isn't there a "clear 
all-sessions-where-capabilities-have-changed" (paraphrasing).

Current behaviour is very disruptive. You change a line of config that you 
might think is a minor change, and BOOM, all your sessions are reset and 
you have a many-minutes outage.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Enke Chen <[email-masked]>
Date: Wed, 26 Apr 2017 22:33:34 -0700

Hi, Mikael:

Please take a look at the "BGP Dynamic Capability" draft:

   https://www.ietf.org/archive/id/draft-ietf-idr-dynamic-cap-14.txt

Regards,  -- Enke

On 4/26/17 10:18 PM, Mikael Abrahamsson wrote:
> 
> Hi,
> 
> disregard the part in my last email about asking why all bgp sessions in a peer group are 
> flapped when one changes the config of that peer group. Let's discuss that here instead.
> 
> If I add "address-family ipv4 multicast" to my peer-group and it wasn't there before, all
> router OSes I know will just reset all the sessions. This is baffling.
> 
> I see two problems:
> 
> 2. Why doesn't it require a command to flap the sessions to adapt to the new configuration?
> If I want to run-through my changed policy-map, I have to "clear soft in" (or out). Why isn't
> there a "clear all-sessions-where-capabilities-have-changed" (paraphrasing).
> 
> Current behaviour is very disruptive. You change a line of config that you might think is a
> minor change, and BOOM, all your sessions are reset and you have a many-minutes outage.
>

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 27 Apr 2017 08:14:07 +0200 (CEST)

>   https://www.ietf.org/archive/id/draft-ietf-idr-dynamic-cap-14.txt

So this is a draft first posted in 2001, with a last revision in 2011.

What's the back story here? This is obviously a well-known understood 
problem then for 16 years already, why don't we still have this in 2017?

---

From: Enke Chen <[email-masked]>
Date: Wed, 26 Apr 2017 23:17:58 -0700

Low demand. The feature request has never bubbled to the top tier for implementation.

-- Enke

On 4/26/17 11:14 PM, Mikael Abrahamsson wrote:
> What's the back story here? This is obviously a well-known understood problem then
> for 16 years already, why don't we still have this in 2017?
>

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 27 Apr 2017 09:44:57 -0400

> problem then for 16 years already, why don't we still have this in 2017?

​Since you asked :) ....

1. Actually number of customers asked for it various vendors however non of
them showed sufficient money so naturally features which are not associated
with significant revenue are not that high on anyone's list (unless BGP
code you are maintaining is your hobby).

2. If we talking intra-domian in most cases ​you go via RRs. And it is a
good practice to configure your RR side either with multiple loopbacks or
even different contexts for each address family so each SAFI is independent
from one another. That way within your domain you can add/delete AFI/SAFIs
without impacting others. Moreover in case of using multiple context (VMs
or LXCs for reflection) you get multithreading across SAFIs for free too.

3. And last in general what really matters is to continue forwarding
packets and not to impact your other peers by BGP session reset. And here
comes quite well supported feature BGP Graceful Restart (RFC4724) It is
applicable for both iBGP and eBGP peers. And honestly many networks still
do not have it enabled even though it has been shipping for long time in
most major BGP code basis.

Kind regards,
Robert.

---

From: Enke Chen <[email-masked]>
Date: Thu, 27 Apr 2017 10:22:27 -0700

Certainly with GR, the need for "Dynamic Capability" is significantly reduced.

Thanks.  -- Enke

On 4/27/17 6:44 AM, Robert Raszuk wrote:
>  
> 
>     What's the back story here? This is obviously a well-known understood problem then for 16 years already, why don't we still have this in 2017?
> 
> 
> ​Since you asked :) ....
> 
> 1. Actually number of customers asked for it various vendors however non of them showed sufficient money so naturally features which are not associated with significant revenue are not that high on anyone's list (unless BGP code you are maintaining is your hobby). 
> 
> 2. If we talking intra-domian in most cases ​you go via RRs. And it is a good practice to configure your RR side either with multiple loopbacks or even different contexts for each address family so each SAFI is independent from one another. That way within your domain you can add/delete AFI/SAFIs without impacting others. Moreover in case of using multiple context (VMs or LXCs for reflection) you get multithreading across SAFIs for free too. 
> 
> 3. And last in general what really matters is to continue forwarding packets and not to impact your other peers by BGP session reset. And here comes quite well supported feature BGP Graceful Restart (RFC4724) It is applicable for both iBGP and eBGP peers. And honestly many networks still do not have it enabled even though it has been shipping for long time in most major BGP code basis. 
> 
> Kind regards,
> Robert.
> 
>

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 27 Apr 2017 16:17:36 -0400

On Thu, Apr 27, 2017 at 08:14:07AM +0200, Mikael Abrahamsson wrote:
> What's the back story here? This is obviously a well-known
> understood problem then for 16 years already, why don't we still
> have this in 2017?

Part of the issue with dynamic caps was that they renegotiated capabilities
in general.  The edge cases of having to rebuild everything dynamically that
you could negotiate and still keep a peering session up was... ugly.

A simple example would be what would happen if you added or deleted the
add-paths capability. 

I know there was discussion at one point to simplify the proposal to simply
to allow new AFI/SAFI to be re-negotiated.

The BGP multisession proposal tried to cover one other piece of the
headache.  Today, if you get a failure in one AFI/SAFI, we tear down BGP.
(You're allowed to "shut it down", but such well behaved failures are
relatively few.)  By having different sessions for the AFI/SAFI, you get
structural segregation of those classes of packet formatting errors.

The issues with multisession, aside from burning precious sockets, is that
there's state that is really shared among the family types.  The question
then becomes how you share that state appropriately when the sessions can
move at different asynchronous paths.

-- Jeff

---

From: Enke Chen <[email-masked]>
Date: Thu, 27 Apr 2017 13:54:10 -0700

Hi, Jeff:

> A simple example would be what would happen if you added or deleted the

That is a reasonable summary.

> to allow new AFI/SAFI to be re-negotiated.

If there is sufficient interest, we can limit the capability to just AFI/SAFI
without much work.  With GR though the interest seems to have dropped.

Thanks.  -- Enke

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 28 Apr 2017 07:08:49 +0200 (CEST)

On Thu, 27 Apr 2017, Enke Chen wrote:

> If there is sufficient interest, we can limit the capability to just 
> AFI/SAFI without much work.  With GR though the interest seems to have 
> dropped.

If someone had asked me 2 days ago "If you have GR enabled on a BGP 
session and add an address family to that session, will this affect 
forwarding when the session restarts" I would have said "YES!"

But from what I'm reading here, I would have been wrong? The session would 
go down but RIB/FIB would be untouched because GR?

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 28 Apr 2017 02:29:37 -0400

Hi Mikael,

Exactly.

For the duration of gr timer you continue to forward till your bgp session
restarts and finishes sending you routes with end-of-rib marker.

You also do not propagate neither new best paths nor withdraws to your
other peers during this restart limiting amount of bgp churn.

Then you compare "stale" routes with fresh ones and adjust the delta both
within your box and to peers.

Just try it out ;)

//R.

On Apr 28, 2017 01:11, "Mikael Abrahamsson" <[email-masked]> wrote:

> AFI/SAFI without much work.  With GR though the interest seems to have

If someone had asked me 2 days ago "If you have GR enabled on a BGP session
and add an address family to that session, will this affect forwarding when
the session restarts" I would have said "YES!"

_______________________________________________
Idr mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/idr

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 28 Apr 2017 09:34:13 +0200 (CEST)

> Just try it out ;)

I have tried it out. From what I remember we had to turn it off because it 
gave us 3 minute outages that we didn't much desire.

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Sun, 30 Apr 2017 22:29:09 +0000

If it goes down with a notification message, you don't get GR.
I haven't tried it, but I bet you get a CEASE.

Thanks,
Jakob.

> -----Original Message-----
> From: Idr [mailto:[email-masked]] On Behalf Of Mikael Abrahamsson
> Sent: Friday, April 28, 2017 12:34 AM
> To: Robert Raszuk <[email-masked]>
> Cc: idr wg <[email-masked]>
> Subject: Re: [Idr] new thread regarding capabilities handling
>

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Sun, 30 Apr 2017 23:06:06 +0000

Right - implementation of
https://www.ietf.org/id/draft-ietf-idr-bgp-gr-notification-10.txt to for
GR to occur with when an notification message is received. Cisco will
support this draft in IOS-XE in release 16.6.1.

Thanks,
Acee