Subject: Re: [bmwg] draft-papneja-bgp-basic-dp-convergence-01

From: bhavani <[email-masked]>
Date: Thu, 05 May 2011 13:02:29 -0700

Ilya,
   thanks for your comments and feedback. Below are the responses to 
address your comments

- Figure 1: s/Err/Trr will be changed/removed

- Figures 2-4: agree it will be easier for correlation, will look into 
this and how related testcase(s) need to be changed if any

- Section 4: the nexthop validation should be same for eBGP&  iBGP and 
line will be removed as its not really a difference

- Section 4: helper router cannot be named simulated because it is a 
real router running same software as the DUT

- Section 4.2: we will address the Route Mixture issue like this - Route 
Mixture and Route Packing are defined in RFC-4098.
    In addition terminology RFC also defines Update Train and Randomness 
in Update Train terms. Since everyone has different
    view of the network/Internet, we want the user to define these 
parameters according to their need/criterion and report them
    for each testcase and use the same when comparing different 
implementations.
    The current draft version already recommends Route Mixture&  Route 
Packing to be reported (sec-6), we will also add
    Update Train&  Randomness in Update Trains charateristics to be reported

- Section 4.4: we will add these

- Section 4.5: To address your concern, we intend to change the section 
to the following.
    It is recommended that all the tests are carried out the same rate 
for ease of comparison between multiple set of devices.
    It is further recommended that the tester may choose the rate at 
which the tests should be carried out based on the least
    common denominator of maximum throughput supported by any interface 
on the DUT.  The rate at which the
    convergence is carried should also be reported in the reporting matrix

- Section 5.1.1: if higher sensitivity is required then stratum-1 can be 
used.

- Section 5.1.2: traffic is actually sent from Emulator to Helper Node. 
The control plane route (ex.route A) is advertised
    by Emulator to DUT whereas the Traffic is actually sent the other 
way from Emulator to Helper Node. Initially traffic
    will not be forwarded on Helper node because the route first needs 
to be received installed by DUT and then advertised
    to Helper which installs it and thats when the traffic would flow. 
we would need to subtract the RIB-in convergence on
    Helper node also for this

- Section 5.1.4: some implementations might have different (like cache 
size) parameters based on iBGP or eBGP and hence
   this testcase. The other scenarios are covered as part of 5.2.1&  5.2.2

- Section 5.3: agreed to your point that we might either see session 
time out or NOTIFICATION sent but even in that failure
   scenario (G) from the testcase would still hold good. Can you also 
let us know which test step suggests that it takes
   less time than anticipated.

- Section 5.7: agreed to your comment, we need to say repeat the test 
with all mandatory well-known attributes

- Section 5.8: step.A should be advertise/withdraw a route. What we want 
to see here is if during switchover you receive a
    new route or receive a withdraw for an existing route, how long will 
it take for the DUT to actually send the update to its
    downstream neighbors.
    Also for your other comments for this testcase, we do not want the 
DUT to react to the route refresh sent by helper node
    when the policy is removed which you are also saying will most 
probably happen. The reason we mentioned pt.S because,
    there might be a time window where the implementation on the DUT 
might be slow in closing the old session which means
    it might end up servicing the route refresh in this scenario and 
then the testcase would only measure the route refresh
    processing on the DUT. Also we can clarify in the testcase that this 
is for scenarios where the original sessions are not
    preserved upon switchover.

regards,
Bhavani

On 3/28/2011 4:49 PM, iLya wrote:
> Hi,
>
> below are some comments on draft-papneja-bgp-basic-dp-convergence-01. 
> Sorry for terse wording, I will elaborate during BMWG session, just 
> thought this may help.
>
> figure 1: s/Err/Trr/ and s/emulator/tester/ or simulator (Err brings 
> associations with error)
>
> Figures 2-4: source and sink ports should probably be on the same 
> tester for easier correlation of number of transmitted and received 
> packets, and possibly time
>
> section 4: typically iBGP routes are installed (by an implementation 
> strictly-conformant to BGP specs) when next-hop is valid & iBGP route 
> doesn't invalidate next-hop & route is best.
>
> "helper routers" should probably be replaced by "simulated routers"
>
> "It is RECOMMENDED that the user may consider advertizing the entire
>   current Internet routing table per peering session using an Internet
>   route mixture with unique or non-unique routes." <- this is not 
> repeatable because everyone has different "Internet" view. We need 
> more accurate description of the test routes that sufficiently close 
> represent typical densely-connected network.
>
> When multiple peers are used it's important to document not only 
> timing betweeb sending routes (I'm not sure that this is practical for 
> general case due to packing differences) but also PATH attribute 
> distribution among peers, variation of PATH attributes between peers. 
> Even better from repeatability is to define "standard" collection of 
> PATH mixture that MUST be used for tests.
>
> s/minimum policy/minimal policy/
>
> section 4.4: baseline run should be done with configurable parameters 
> set to a) default values specified by respective RFC; b) 
> platform-specific baseline - set parameters to values that are default 
> for given platform; and c) values expected on operational network
>
> add to basic parameters: Max TCP Window size (or fully dynamic if 
> supported) and MTU.
>
> Section 4.5: perhaps BGP test specs should require tests at specific 
> set of speeds (so operators know if vendor results are comparable)
>
> Section 5.1.1: Step E is impractical because clocks of DUT and the 
> tester are not sufficiently sync'ed (NTP stratum 2 is not good enough 
> for high-speed links, stratum 1 is extremely difficult to implement), 
> and therefore reference to RCV-Rt-time in step H is meaningless. Also, 
> there are few different types of Average.
>
> Section 5.1.2: same as 5.1.1 plus: not clear why traffic would ever 
> flow towards helper node if route advertised by Tester/Simulator and 
> what is "Route Egress Interface toward peer-X" - steps G,H,K are 
> confusing. Also, Testers/Simulators typically use "DUT address" as 
> default gateway to forward traffic towards anything for which they 
> don't have specific route, this means if source port of Tester sends 
> traffic prior receiving route from DUT (but DUT already installed 
> route in FIB), traffic will be successfully received at the sink port. 
> Need a methodology to overcome this problem.
>
> Section 5.1.4: iBGP convergence should focus on detecting next-hop 
> changes/failures both due to directly attached link failure as well as 
> remote failure/changes. As currently specified test does not seem to 
> have relevance (or alternatively - this section measures something 
> else but not iBGP convergence).
>
> Section 5.3: "software ajacency down" on Tester will likely be 
> reported immediately to DUT. In real network there is usually 
> communication problem (but different from link failure) from Tester to 
> DUT that results in DUT not receiving KEEPALIVEs and timing out 
> session with Tester (or in reverse) and NOTIFICATION is sent. This 
> process takes longer than described in this section, but of far higher 
> practical value. Alternatively: this section describes something else 
> (e.g. session termination by peer or even Hard Reset as in section 
> 5.4.2) but not adjacency failure.
>
> Section 5.7: Significantly different results may be observed if route 
> in step H has better local-pref (on iBGP session) compare to case when 
> it has better AS_PATH. For next-hop - when tie-breaking on cost to 
> next-hop in some scenarios some platforms have O(1) convergence for N 
> prefixes. To ensure comparable results this section must spell out 
> which attributes should be better (or require several runs for various 
> attributes).
>
> Section 5.8: This section does not seem to address Graceful Restart. 
> For example, Route-Refresh if sent by non-DUT device will either not 
> reach DUT (on old session, which is closed) or DUT will not react (per 
> RFC4724) on it until it sees EoR from non-DUT (if RR sent on new 
> session). Additionally, some implementations may even preserve 
> original session (I'm not aware of existence of such, but they're 
> feasible) and nothing happens at all during switchover. It's not clear 
> what was intention of the test in this section.
>
> Kind regards,
> iLya
> _______________________________________________
> bmwg mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/bmwg
>

---

From: "iLya" <[email-masked]>
Date: Mon, 9 May 2011 17:19:25 +0200

Hi Bhavani,

thanks for your clarification. For many points I'll wait to see -02, but 
let's discuss few of them here (inline).

--------------------------------------------------
> - Section 4: helper router cannot be named simulated because it is a real 
> router running same software as the DUT
>

I believe it's invalid to use a device of unknown behaviour to measure 
performance of another unknown device - you never can be certain whether 
observed values are due to DUT itself or due to "helper" router. Also, the 
presence of "helper" router makes setup more complicated than necessary. In 
my opinion we should aim that all performance test could be executed using 
just two devices: DUT and simulator. The latter can simulate various 
topologies and advertise all necessary attributes.

> Mixture and Route Packing are defined in RFC-4098.

Before posting my original comments I did check that RFC, but it contains 
definition of only Route Mixture term and not what that Route Mixture looks 
like.

> - Section 5.1.1: if higher sensitivity is required then stratum-1 can be 
for a router clock to run at stratum-1 it has to sync from stratum-0, that 
is from an atomic clock or GPS receiver. I don't think majority of routeing 
platforms (except for PC's running general-purpose Unix and a routeing 
daemon) there is option to connect either atomic clock or GPS receiver. 
That's why I've said that step E is impractical. And not to be just a 
nay-sayer, the alternative is to construct methodology in such a way that 
same clock is used for timing events. That's where using simulator instead 
of a helper router comes handy.

>   this testcase. The other scenarios are covered as part of 5.2.1&  5.2.2
there's no doubt that test described in 5.1.4 provides useful info, however 
it's not iBGP-specific test. I would combine sections 5.1.3&4 into single, 
call it "general convergence" and add 'ibgp/ebgp' as one of the test 
parameters. But it still leaves missing the case that targets explicitly 
iBGP - disappearance or change of cost of next-hop.

> - Section 5.3: agreed to your point that we might either see session time 
> out or NOTIFICATION sent but even in that failure
>   scenario (G) from the testcase would still hold good. Can you also let 
> us know which test step suggests that it takes
it's not that described step takes less or longer than anticipated, but that 
5.3 doesn't cover the case where peer was not shut but rather communication 
to the peer got broken yet link didn't go down. In this case, unless BFD is 
used, DUT would have to wait until holdtimer runs out. This is the time 
which takes longer than described by the test.

> - Section 5.7: agreed to your comment, we need to say repeat the test with 
> all mandatory well-known attributes
>

I think we need to be more explicit than just saying "all mandatory 
well-known".

>    new route or receive a withdraw for an existing route, how long will it 
> take for the DUT to actually send the update to its
>    downstream neighbors.

not before downstream neighbor has sent EOR (not allowed otherwise), so 
you're effectively measuring performance of the downstream neighbor rather 
than DUT.

>    Also for your other comments for this testcase, we do not want the DUT 
> to react to the route refresh sent by helper node
>    when the policy is removed which you are also saying will most probably 
> happen. The reason we mentioned pt.S because,
>    there might be a time window where the implementation on the DUT might 
> be slow in closing the old session which means

so you mean an implementation that would have two sessions in ESTABLISHED 
state towards the same neighbor (modulo multisession)? sounds like broken 
implementation to me and should be addressed in "bgp comformance 
methodology" rather than "performance".

Since the time of original post, one more point came to my mind - on several 
modern platforms forwarding to a destination can be restored MUCH before BGP 
or IGP converges (by using pre-computed backup path independently from 
BGP/IGP convergence process), this fact will invalidate some of the tests 
for such platforms. This also makes presence of the helper node serious 
problem because such feature may work differently depending on the placement 
of the router, so DUT itself may recover forwarding functionality much 
quicker than helper node even though they're same platform and run same 
software. Yet another argument for getting rid of the helper node.

Kind regards,
iLya