Subject: [bmwg] WG Last Call: FIB benchmarking methodology.

From: Kevin Dubray <[email-masked]>
Date: Fri, 14 Feb 2003 10:45:03 -0500

BMWG'ers:

A WG Last Call period for the Internet-Drafts regarding a
Methodology for Forwarding Information Base (FIB) based Router
Performance:

           <draft-ietf-bmwg-fib-meth-01.txt>

will be open from 14 February 2003 until 14 March 2003.

This is the proposed methodology companion document to RFC 3222.
Note that this I-D has been out there for a while and has
received little commentary.

So, please weigh in on whether or not you feel this internet-draft
should be given to the Area Directors for consideration in
progressing the I-D to an Informational RFC.  Send your comments
to this list or [email-masked].

A URL for the Internet-Drafts are:
http://www.ietf.org/internet-drafts/draft-ietf-bmwg-fib-meth-01.txt

-Kevin

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: "Dawit Birhanu" <[email-masked]>
Date: Fri, 14 Feb 2003 10:17:13 -0800

A number of test equipement verndors have implemented RFC2544 benchmarking
suite to their test. Router vendors as well as customers have greately
benefited from this. However, we have seen some issues with these tests with
respect to performance and latency testing. The purpose of this e-mail is to
initiate a discussion on this issues.

1. RFC2544 Throughput Testing

RFC 2544 defines throughput as "the fastest rate at which the count of test
frames transmitted by the DUT is equal to the number of test frames sent to
it by the test equipment." The issue that we have seen is that the no-drop
rate usually overestimates the actuall performance of the DUT. The
measurement error depends on the duration of the test and size of buffering
in the DUT. However, RFC-2544 does not specify nor require the reporting of
test duration and max/min/avg latency measurements during throughput
testing. Note that the test duration T, the minimum latency Dmin, the
maximum latency Dmax and the measurement error due to buffering Eb can be
related as follows:

Eb = (Dmax - Dmin)/T

This is assuming that the latency variation is mostly attributed to
buffering.

There are two possible solutions to this problem:

a. Redefine throughput as "the fastest rate at which count of test frames
transmitted by DUT is qual to the number of test frames sent to it by the
test equipment without exceeding a specified maximum latency. The test
duration should be at least 1000 (or 100) times the specified maximum
latency".

b. Require the vendor to include Test duration and maximum latency in the
test report.

2. RFC2544 Latency Testing

RFC2544 requires that latency be measured while sending traffic at the
no-drop throughput rate. This in most cases leads to queue buildup in the
DUT, resulting in extemely high latency measurement.

One possible solution for this problem is to require that latency be
measured at 90%, 95% and/or 99% of measured throughput.

Thanks,
Dawit Birhanu
[email-masked]

---

From: "Dawit Birhanu" <[email-masked]>
Date: Fri, 14 Feb 2003 10:17:13 -0800

Eb = (Dmax - Dmin)/T

Thanks,
Dawit Birhanu
[email-masked]