Subject: RE: [bmwg] Accelerated Stress - Comments from IETF 60

From: [email-masked]
Date: Wed, 11 Aug 2004 09:51:27 -0400

Al,

Item 9 does not satisfy that suggestion.  The point you raise below is a
good one.  Do the benchmarks introduced in the -03 Terminology satisfy the
ability to compare different platforms?

Scott

-----Original Message-----
From: Al Morton [mailto:[email-masked]]
Sent: Tuesday, August 10, 2004 11:28 PM
To: [email-masked]
Subject: Re: [bmwg] Accelerated Stress - Comments from IETF 60

At 08:18 PM 08/09/2004, [email-masked] wrote:
>...
>METHODOLOGY
>6. Add test case for establishment of new EBGP peer
>7. Add test case for configuration/removal of BGP policy
>8. Use SYN Flood attack for the DoS Attack test case
>9. Add steps to each test case to accurately describe the conditions of
each
>specific test case
>Scott
Maybe the last item covers it, but my parting comment was to
think hard about how to foster cross-platform comparisons in
the methodology. Configurations have many platform-specific
aspects, so how do we bridge the diffs to obtain insightful
benchmarks?

Al
(as wg participant) 

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: Al Morton <[email-masked]>
Date: Wed, 11 Aug 2004 10:36:32 -0400

>ability to compare different platforms?

This isn't a problem with the benchmarks themselves
(how do you go wrong with Forwarding Rate?), it's more
about ensuring a level playing field with the setup/config.
With so many processes and protocols in operation, there are bound to
be differences in the default values across platforms, and the defaults
supplied for non-benchmarking purposes could affect the results.

Or, are *all* the benchmarks normalized to the DUT's stable state
performance?  I don't think that's the case, here.
I'm just trying to see a clear picture of how all this will provide
insight when making comparisons.

Al