Subject: [bmwg] URL to BGP covergence methodology issues slides.

From: Kevin Dubray <[email-masked]>
Date: Fri, 07 Dec 2001 11:48:06 -0500

One of the agenda items for the BMWG session at SLC is to
solicit input with regards to issues facing the convergence 
benchmark methodologies.

The principals offer a view of the issues here:

   http://www.lepp.com/IETF/BGPconv.ppt

It would be great if you could bring your input to the
meeting!  (For those unable to attend, your feedback is
welcome right here on this list.)

-Kevin

_______________________________________________
bmwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/bmwg

---

From: "David Newman" <[email-masked]>
Date: Sat, 8 Dec 2001 02:22:59 -0800

nice sound effects :)

Besides the factors mentioned in the slides, some other possible issues to
consider in building test tables:

--prefix length distribution?
--LOCAL_PREF in use?
--contiguity of prefixes? (aggregation is good in production, bad in
testing)
--for flapping, contiguity of prefixes withdrawn or readvertised (big impact
on forwarding in some implementations)

Also, in measuring convergence times, obviously the number of
boxes/interfaces matters a lot in any convergence measurement. I'm sure the
presenters will cover this; I just didn't see it in the slides.

dn

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of Kevin
> Dubray
> Sent: Friday, December 07, 2001 8:48 AM
> To: [email-masked]
> Subject: [bmwg] URL to BGP covergence methodology issues slides.
>
>

---

From: "David Newman" <[email-masked]>
Date: Sat, 8 Dec 2001 02:22:59 -0800

nice sound effects :)

dn

---

From: Padma Krishnaswamy <[email-masked]>
Date: Sat, 8 Dec 2001 08:39:08 -0500

thank you. Sound credits should go to HCB. 
Your pts are well taken; the issues you raise were  generally
lumped under the 'route mix' concept
but not very explicitly called out in the slides. wanted to elicit response
from the working group, such as your comments here, for instance.

The number of boxes/i/fs would be variants on the no of peers and topology.

Have you any rules of thumb as to this: ie, if testing core routers, how many peers would you consider representative?

One other thing,we're still not including forwarding. The reason is we want
to clearly define the aspects of  convergence in the routing plane before
the next step- combining the convergence measurements to include the data plane-
which could be the forwarding table plus any other ops relegated to that dimension. The reason is to isolate at least conceptually the different effects,
 so we can define tests which might measure the same phenomenon in the least
implementation dependent way.

Padma

---

From: Padma Krishnaswamy <[email-masked]>
Date: Sat, 8 Dec 2001 08:39:08 -0500

Padma

---

From: "David Newman" <[email-masked]>
Date: Sat, 8 Dec 2001 08:03:28 -0800

> Have you any rules of thumb as to this: ie, if testing core
> routers, how many peers would you consider representative?

Of course, the question here is "representative of what?" I don't know that
there is any one number or range that is meaningful for every user, or even
for most users. I have a few numbers but they're 2-3 years old and I don't
know if they're still accurate.

Maybe Randy or other folks with ops experience can comment here.

> which could be the forwarding table plus any other ops relegated
> to that dimension. The reason is to isolate at least conceptually
> the different effects,

Totally agree. BTW, in previous tests I've done of convergence, forwarding
*was* the metric. Your approach of isolating the control-plane function is
better.

I'm also glad to see you do plan to add forwarding measurements once the
control-plane metrics are nailed down.

dn
(who regrets not being in SLC because he's in Asia lecturing on this stuff)

---

From: "David Newman" <[email-masked]>
Date: Sat, 8 Dec 2001 08:03:28 -0800

dn

---

From: "Howard C. Berkowitz" <[email-masked]>
Date: Tue, 11 Dec 2001 09:53:55 -0500

>Maybe Randy or other folks with ops experience can comment here.

Do note there are some representative route mixes for various kinds 
of router (core, interprovider, customer, etc.) in Appendix A of the 
methodology document.

The concern about number of interfaces/peers, however, gets very 
touchy because we are getting right on the edge of where the 
salesdroids start trying to manipulate. Can there be -- and I 
honestly mean this as a question -- a consensus on this?

An alternative is simply requiring the number of interfaces and peers 
to be a part of the test report, so the market/customer will demand 
comparable reports on competitive routers.

---

From: "Howard C. Berkowitz" <[email-masked]>
Date: Tue, 11 Dec 2001 09:53:55 -0500