Subject: [RRG] BGP Graceful Restart (was Re: Geoff Huston's article ...)

From: [email-masked]
Date: Wed, 20 Jun 2007 08:34:56 -0400

For the purpose of damping BGP or reducing its churn, 
another important tool that should be quite useful is the 
BGP graceful restart (BGP-GR) mechanism (RFC 4724).
I was at the recent NANOG-40 where in the Peering BOF
session it was discussed that BGP restart is an available
feature generally in BGP routers but it is turned off
by default, and it appears that many smaller ISPs do not 
activate it. It would be interesting to know (if there is
data available) how often BGP session restarts occur
(due to software upgrades or any other reasons).
A BGP restart at a BGP speaker causes its peer to withdraw 
routes learnt from that BGP speaker (in the absence of BGP-GR).
This produces churn and can also trigger Route Flap Damping (RFD)
(two or more hops away from that BGP speaker), which adds 
substantial delay to convergence or re-convergence.
These problems are avoidable by activating BGP-GR on all or most 
BGP peering sessions. BGP-GR can also be an effective countermeasure
against deliberate attacks that seek to cause BGP session restarts.
Some references that discuss this topic further:
http://csrc.nist.gov/publications/drafts/800-54/Draft-SP800-54-version2-
Jun2007.pdf
(see BGP-GR overview and discussion in Section 5.1)
http://www.antd.nist.gov/~ksriram/BGP_Security_Sriram_IEEE_JSAC.pdf
(see Section V)

Sriram

Quoting Geoff Huston <[email-masked]>:

> Robin Whittle wrote:
> > Geoff Huston has a new article "Damping BGP":
> > 
> >   http://www.potaroo.net/ispcol/2007-06/dampbgp.html
> > 
> > He proposes methods of identifying updates which are usually
> > associated with path hunting.  Typically this is one or multiple
> > announcements of a longer path for the prefix, followed by the
> > withdrawal of the prefix.  I understand from this that he
> > supports "path length damping" as described by Tony Li:
> > 
> >    http://tools.ietf.org/html/draft-li-bgp-stability
> > 
> > How such filtering would work when all routers implement it I am
> > not sure, but it seems like a good idea.
> 
> The withdrawal would propagate, or the path would stabilise. One way to 
>   view it is as a more selective form of Min Route Advert Interval Timer.
> 
> 
> > I learnt some interesting things, including that BGP routers
> > often use TCP to throttle the updates from peers to a rate they
> > can handle.  One reason this is done is to prevent local input
> > queues from taking up too much memory.
> 
> Thank you John Scudder for clearly explaining this to me earlier this year.
> 
> > 
> > Meanwhile, the peers perform "output queue compression".  They
> > look into the queue of updates which are waiting to be sent and
> > remove any messages which have been rendered incorrect by events
> > which transpired since that message was put in the queue.
> 
> Thank you John Scudder for clearly explaining this to me earlier this year.
> 
> > 
> > There is also a diagram depicting path hunting, and graphs
> > depicting the statistics of updates, based on research in April
> > this year.
> > 
> >   "10% of announced prefixes being responsible for 53% of all
> >    routing updates, and the busiest 1% of prefixes responsible
> >    for 24% of the routing updates for the month."
> > 
> >   "It can’t be the case that more than a million routing updates
> >    actually reflect true underlying changes in topology of the
> >    network, given that these one million updates only refer to
> >    2,000 prefixes."
> > 
> > I think the time analysis of the updates would be of interest to
> > anyone concerned about BGP stability.
> 
> I would hope so. A significant proportion of the BGP traffic appears to 
> be concerned with a relatively small number of "pathologies" where BGP 
> itself is the amplifier.
> 
>     Geoff
> 
> 
> 
> 
> --
> to unsubscribe send a message to [email-masked] with the
> word 'unsubscribe' in a single line as the message text body.
> archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg
> 

--
to unsubscribe send a message to [email-masked] with the
word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg

---

From: Geoff Huston <[email-masked]>
Date: Thu, 21 Jun 2007 04:20:45 +1000

> BGP graceful restart (BGP-GR) mechanism (RFC 4724).

All depends on what you see as being the major factors that contribute 
to the BGP processing load in today's routing environment. Graceful 
restart is very effective, and it should be deployed ubiquitously, but 
from where I sit in the routing mesh the occasional spike I see in the 
update rate does not strike me as being a major contributor to the BGP 
update rate. Of course this is a subjective rather than a qualitative 
response and your mileage in your neck of the woods may vary!

Geoff

---

From: Tony Li <[email-masked]>
Date: Wed, 20 Jun 2007 11:39:48 -0700

On Jun 20, 2007, at 11:20 AM, Geoff Huston wrote:

> [email-masked] wrote:
>> For the purpose of damping BGP or reducing its churn, another  
>> important tool that should be quite useful is the BGP graceful  
>> restart (BGP-GR) mechanism (RFC 4724).
>
> All depends on what you see as being the major factors that  
> contribute to the BGP processing load in today's routing  
> environment. Graceful restart is very effective, and it should be  
> deployed ubiquitously, but from where I sit in the routing mesh the  
> occasional spike I see in the update rate does not strike me as  
> being a major contributor to the BGP update rate. Of course this is  
> a subjective rather than a qualitative response and your mileage in  
> your neck of the woods may vary!

As Geoff points out, the best that GR could do is to alleviate some  
churn due to software restarts.  Real link failures or hardware  
failures are still going to result in churn.

There's also a reliability and confidence issue with GR.  The  
presumption with GR is that a router advertising its restart  
capabilities will have a forwarding plane that will continue to  
reliably forward during a failure in the control plane.  If it does  
not do this, then it will either blackhole or misroute traffic, which  
is obviously not good.  Many carriers don't yet have the confidence  
is the implementations to put this into production.  Hopefully, of  
course, this will change.  However, I don't think we should wait for it.

Regards,
Tony

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 20 Jun 2007 15:11:56 -0400

On Jun 20, 2007, at 2:39 PM, Tony Li wrote:
> The presumption with GR is that a router advertising its restart  
> not do this, then it will either blackhole or misroute traffic,  
> which is obviously not good.

Note that for GR between IBGP peers, this potential issue is  
reduced.  That's because a control plane failure which is serious  
enough to affect the forwarding plane is likely to also take out the  
IGP.  Since IBGP routes are typically resolved through the peer's  
loopback address which is learned through the IGP, an IGP failure  
will have the effect of invalidating dependent IBGP routes,  
regardless of their GR status.

But this is quite a tangent.

--John

---

From: Danny McPherson <[email-masked]>
Date: Mon, 25 Jun 2007 07:38:51 -0600

On Jun 20, 2007, at 1:11 PM, John G. Scudder wrote:

>> reliably forward during a failure in the control plane.  If it  
>> does not do this, then it will either blackhole or misroute  
>> traffic, which is obviously not good.
>>
>
> enough to affect the forwarding plane is likely to also take out  
> the IGP.  Since IBGP routes are typically resolved through the  
> peer's loopback address which is learned through the IGP, an IGP  
> failure will have the effect of invalidating dependent IBGP routes,  

Unless, for example, an implementation were to use some different
handling for IP-based stuff (e.g., BGP), versus, say, IS-IS.  You might
laugh but I had this occur a couple years back (several times, mind you,
under different conditions, on a very popular router vendor/model) when
IP forwarding fails but IS-IS would stay up, so next hops were reachable
and packets blindly discarded.  The interesting bit was that IP  
forwarding
was only failing on certain line cards, but the result was the same.

Of course, the architectural flaw was quickly hacked around, but it was
a reality and I wouldn't be surprised to see something of this sort
recur, in particular when employing non-IP IGPs.

-danny