Subject: RE: [bmwg] Device Convergence Benchmarking

From: "Marianne Lepp" <[email-masked]>
Date: Sun, 17 Mar 2002 16:01:47 -0800

This would be similar to the tests I ran at Genuity with a complete 4
node network (6 links, one ethernet, 5 OC3s) to understand an anomaly
that turned up in snmp data.  On running the test, I found some very
interesting behavior, with all convergence times gated by timers.  
 
I like that 4 node network and would suggest it for a BGP study, one
step wider than the benchmarking of a single BGP device, that we are
talking about in our draft.
 
Marianne

-----Original Message-----
From: Elwyn Davies [mailto:[email-masked]]
Sent: Tuesday, March 12, 2002 1:15 PM
To: Kevin Dubray; [email-masked]
Subject: RE: [bmwg] Device Convergence Benchmarking

Hi. 

I think the approach which have so far taken with the BGP benchmarking
has concentrated our minds on the difficulty of really tying down the
test conditions and thinking through the variables that can affect
convergence in an individual router.

However, it also clear that this is only a (small) part of the story.  I
came across this paper from Tim Griffin (
http://www.research.att.com/~griffin/icnp2001.html) which is about
simulated BGP and (mostly) the effects the value of MIN_ROUTER_ADVERT
timer on the withdrawal behaviour of a network.  It would be intriguing
to try to replicate these results with real routers:  This is clearly
about device convergence as it can be measured on a single device but it
needs the surrounding CLOSED network in order to demonstrate the
effects.  Indeed there may be additional effects due to the truly
asynchronous arrival of packets on different interfaces that are not
necessarily seen in a simulation.

It should be possible to combine the practical insights which we have
had into the characterisation of update packet trains and the sorts of
semi-theoretical insights into spcific network topologies such as
Griffin's work provides.

Regards, 
Elwyn Davies 

> -----Original Message----- 
> From: Kevin Dubray [ mailto:[email-masked]] 
> Sent: 06 March 2002 15:47 
> To: [email-masked] 
> Subject: [bmwg] Device Convergence Benchmarking 
> 
> 
> Currently, we have two efforts in the BMWG regarding 
> "convergence":  one WG item on BGP device convergence and 
> a proposed work item on OSPF device convergence. (The last call 
> on the new work proposal recently closed.) 
> 
> The WG chose a device-based division of labor in our first attempt to 
> characterize convergence because of the complexity and breadth 
> of the subject.  It's generally held in the WG that characterizing 
> device *GP convergence is a "good thing." 
> 
> What we're learning is while the collection methodology is usually 
> straightforward, specifying the collection context is not. 
> This is significant because test context (i.e., the simulated network 
> scenario and ensuing test stimuli applied to the DUT) helps gives 
> meaning to the test results. 
> 
> And implied meaning appears central to the some of the critique 
> regarding the effort.  Some questions are: 
>    
>    Is the current division of labor (singular device characterization)

>    so constrained as to have little utility? 
>    Will these tests yield insight into operating networks? 
>    Are the suggested methodologies applicable to characterizing 
>    "convergence" in live networks? 
>    Is lowest "convergence time" always best, or are there 
>    qualitative classes of responses? 
>    What's the significance of these classes of metrics when operators 
>    often adjust protocol timers to trade-off convergence time for 
>    network stability? 
>    Are these results applicable to a single device only or should 
>    the results form a basis for comparison across different devices 
>    of the same type? 
>    How should the reader of these benchmarks apply these results? 
>    What significance should the network operator put on these 
>    results? Is this the same for the enterprise network admin? 
> 
> While sometimes implied, these issues don't appear to be directly 
> addressed by the related work. 
> 
> The BMWG has generally not attempted to define goodness for its 
> benchmarks. That said, it might be useful given the network-wide 
> implication of the term "convergence,"  for the BMWG to 
> provide stronger guidance as to the applicability, use, and limits 
> of these particular benchmarks. 
> 
> Thoughts? 
> 
> -Kevin 
> 
> _______________________________________________ 
> bmwg mailing list 
> [email-masked] 
> https://www1.ietf.org/mailman/listinfo/bmwg 
>

---

From: "David Newman" <[email-masked]>
Date: Mon, 18 Mar 2002 09:42:29 -0800

> I like that 4 node network

Agreed. In fact, it could be argued that four nodes is the minimum necessary
to exercise a routing protocol:

--one node is an island
--two nodes and a cable cut is two nodes off the air
--three nodes and a cable cut is a dumb, automatic form of routing, since
there is only one alternate path
--four nodes and a cable cut forces arbitration, so now we are beginning to
do something like routing

dn

---

From: "David Newman" <[email-masked]>
Date: Mon, 18 Mar 2002 09:42:29 -0800

--one node is an island

dn