Subject: RE: [bmwg] Some issues in RFC-2544

From: Jim McQuaid <[email-masked]>
Date: Tue, 18 Feb 2003 08:21:11 -0500

First of all, 2544 does deal with the issue of how long a receiver should
wait in section 23; "Wait for two seconds for any residual frames to be
received."

Section 24, titled "Trial Duration" is pretty clear in my opinion.  Most
importantly, it states the GOAL of the test duration, "The aim of these
tests is to determine the rate continuously supportable by the DUT."  That
is a lot better than requiring some specific time, because, obviously, for a
given test time, there is a buffer size large enough to "fool" the test.
This section further states that the duration SHOULD be at least 60 seconds,
which is a pretty large amount of traffic to buffer at any common network
speed today, I think.  (Also a long time to age a frame . . .)  Finally,
there is a reference to "binary search" for throughput, which in practice
means starting with the highest possible value of throughput we can test for
/ expect and observing {typically} some frame loss, than working down using
what I will call a gross interval, until you have  bracketed the frame rate
for throughput, followed by a finer-grained repeat of the same thing.  By
starting at the top, so to speak, we establish that there is a frame loss
condition.  If the device is capable of sustained wire speed, then our only
focus would be the test duration.

I haven't looked closely at 2544 for a while, but it seems to be sufficient
already re: your concerns about duration, waiting, etc.

Jim McQuaid

-----Original Message-----
From: Dawit Birhanu [mailto:[email-masked]]
Sent: Tuesday, February 18, 2003 2:35 AM
To: David Newman; John Dawson
Cc: [email-masked]
Subject: RE: [bmwg] Some issues in RFC-2544

> Reinterpreting 2544 also takes us down the path of trying to redefine
> existing terms. it's taken us long enough to make clear that throughput !=
> forwarding rate; let's not go changing those meanings now.
The problem with the current definition "throughput = no drop rate" is that
it
can result in dramatically different measurement values depending on the
duration
of the test. 2544 does not specify the duration of the test. Also it does
NOT specify how long the receiver must wait to receive the last packet after
the transmitter has completed transmitting.

>
> As Jerry noted, the dsterm draft introduces a new term, delay, that
> should help address John's concerns about latency.
>
> dn
>
> _______________________________________________
> bmwg mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/bmwg
>

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg
_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: "Dawit Birhanu" <[email-masked]>
Date: Tue, 18 Feb 2003 10:00:16 -0800

Jim,

I rechecked RFC2544 and it has both 2 second residual time and 60 second
test duration.  (my mistake).

But the main problem that I have is that depending on the size of buffering,
two routers with identical forwarding capability could result in different
performance measurment values. Note that 2 second residual time with 60
second test duration could potentially result in overestimation of the
throughput performance by upto 3.3% (2/60). This suggests that, to decrease
the over-estimation we have to decrease the residual time or increase the
test duration.

-dawit

> From: Jim McQuaid [mailto:[email-masked]]
> Sent: Tuesday, February 18, 2003 5:21 AM
> To: [email-masked]; David Newman; John Dawson
> Cc: [email-masked]

---

From: "Dawit Birhanu" <[email-masked]>
Date: Tue, 18 Feb 2003 10:00:16 -0800

Jim,

-dawit