Subject: Re: [bmwg] draft-varlashkin-router-conv-bench-00

From: Jay Karthik <[email-masked]>
Date: Tue, 01 Nov 2011 12:54:04 -0400

Ilya, thanks for all the clarification.

Please see inline - got an additional comment.

Thx,
Jay

On 10/25/11 3:17 PM, "Ilya Varlashkin" <[email-masked]> wrote:

> Hi Jay,
> 
> thank you for your comments. Please see responses inline.
> 
> On Oct 25, 2011, at 02:07 , Jay Karthik wrote:
> 
>> Rajiv, Ilya,
>> 
>> Could you include what is out of scope for this document ? As I read this
>> initial version, I am reminded of the failure detection methodology, ECMP,
>> concurrent failures, span failure (A single span cut in the fiber network
>> can cause multiple link failures at other layers).
>> 
> 
> There are failure scenarios described in Section 5. The intention was to limit
> tests to those scenarios because they're clear in regards to what's failing
> and what reaction to expect. If you believe that some scenarios should be
> changed or more scenarios need to be added, we could discuss this possibility.
> Please note that the goal of the document is to produce a methodology that
> depends only on DUT performance, i.e. performance of other systems (however
> good or bad) either doesn't affect results at all or can be obtained prior the
> tests and subtracted from overall performance to obtain DUT performance alone.
> Also, this document should be treated as complementary to
> draft-papneja-bgp-basic-dp-convergence-02, which contains more test cases.
> 

Snippet from your ID -
<start-quote>
   When a failure occurs on the network a router needs to:

   1.  select new best path so that the packets, which already arrived
       to the router, can be forwarded

   2.  let other routers know about new network state so they can find
       new best path from their perspective

<end-quote>

Convergence is dependent on how rapidly you detect a failure. You explain
what a node needs to do when a failure occurs but I am wondering if you
should add more color on how you are detecting it. Depending on whether the
failure is a local or remote failure & your detection mechanish - different
protocols/layers come in to play & the expected behavior on the DUT might
change accordingly. Do you agree ?

---

From: "iLya" <[email-masked]>
Date: Tue, 1 Nov 2011 18:40:44 +0100

Hi Jay,

--------------------------------------------------
From: "Jay Karthik" <[email-masked]>
Sent: Tuesday, November 01, 2011 5:54 PM
To: "Ilya Varlashkin" <[email-masked]>
Cc: "Rajiv Papneja" <[email-masked]>; "Scott Poretsky" 
<[email-masked]>; "IETF BMWG" <[email-masked]>
Subject: Re: [bmwg] draft-varlashkin-router-conv-bench-00

> change accordingly. Do you agree ?

Yes, as will also different operating system release running on the same DUT 
with the same config, or different network interfaces used for the test. We, 
authors, had in mind to allocate place in the reporting section to document 
all this "colours". Before we dive into it we would like to sort out 
methodology itself. But yes, I do agree that we need to ensure that tests 
are repeatable and results are comparable.

Kind regards,
iLya