Subject: Re: [bmwg] BMWG Last Call: draft-ietf-bmwg-igp-dataplane-conv-***-02

From: Al Morton <[email-masked]>
Date: Thu, 03 Jun 2004 17:06:19 -0400

IGP Dataplane-ers,

As a BMWG participant, a few comments on
At 11:50 AM 05/10/2004, somebody named Al Morton wrote:
>BMWG ... Last Call ...
>Terminology for Benchmarking IGP Data Plane Route Convergence
>http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-igp-dataplane-conv-term-02.txt

I read Bob Holley's comment that Figure 1 was unclear,
and I wonder if some of the difficulty was related to formatting.
When I print this draft, some of the lines in the figure are
shifted and the sloping lines don't line-up. I know it
looks right on screen, but I suspect tabs or other problem...

Section 3.14 Packet Sampling Interval
The text says rate, but the term is an inter-polling interval,
the reciprocal of rate.

To illustrate, this or similar wording would be OK with me:

Definition: The time interval for counting packet arrivals, possibly
to determine the Forwarding Rate. For example, the interval between
readings of a packet counter.

Al

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: Al Morton <[email-masked]>
Date: Sun, 06 Jun 2004 21:54:20 -0400

At 11:50 AM 05/10/2004, Al Morton wrote:
>BMWG:
>
>A WG Last Call period for the Internet-Drafts titled:
>
>Benchmarking Methodology for IGP Data Plane Route Convergence
>http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-igp-dataplane-conv-meth-02.txt
>...

A few comments on the Methodology, again as a participant:

Though the figures printed just fine in the version-01,
they all suffer the same miss-alignment I mentioned in
the term memo comments.

On section 3.2.4, we had the following discussion (7/9/04):

>>> > [Scott]
>>> > 2. Convergence Timers
>>> > The Methodology draft "Test Considerations" section currently states 
>>> to set
>>> > Convergence Timers to minimum value. It has been requested that we 
>>> include
>>> > suggested values. We propose that the values appear as follow:
>>> >
>>> > SONET Failure Indication Delay (<10ms, almost immediate)
>>> > IGP Hello Timer (1 sec)
>>> > IGP Dead-Interval (3 sec)
>>> > LSA Generation Delay (0, immediate)
>>> > LSA Flood Packet Pacing (0, immediate)
>>> > LSA Retransmission Packet Pacing (0, immediate)
>>> > SPF Delay (0, immediate)
>>> >
>>> > Is there agreement, concern, or questions for these changes to be 
>>> made for
>>> > the 01 versions of the drafts? Are there additional issues that should be
>>> > addressed?
>>> >
>>
>>[Al] Yes.  I agree that the timer values should be reported with
>>the results.  But, if we require very low specific values,
>>we run the risk of precluding benchmarks for some products.
>>(I have *not* surveyed the configuration ranges for all manufacturers.)
>
>[Scott] This is true and certainly a concern.
>
>>[Al] If a configuration interface places limits on timers, the designers
>>may have had a good reason to do so.  Do we require them to remove
>>these safety limits in order to be benchmarked?
>
>[Scott] We should not.
>
>>[Al] The proposal to report timer values raises another issue:
>>The Methodology Draft has no Reporting Formats specified.
>>Perhaps we can do something creative with the reports,
>>where we normalize-out relevant timer values and simply report
>>the additional convergence time? (thinking out loud here)
>>Then we don't encourage designers to allow configurations
>>that might be unstable just to perform well on the benchmarks.
>>This is worth some investigation, IMO.
>
>[Scott] I agree.  This is a good idea.  The co-authors will propose a 
>reporting format that facilitates the calculations in section 5.2.  The 
>"recommended" values can still be provided for guidance.
>
>>thanks for starting discussions, more comments to follow,
>>Al

and I think that section 3.3, Reporting Format, was the result.
One of the suggestions still applies: reporting format with normalization,
especially useful when the recommended values cannot be set.
The raw and normalized results could appear under "Results".
It would be useful to provide the equations for normalization,
(this could help if timers have slightly different names
across vendors, too).

(Editorial comment: ECMP should be spelled out at first use, sec 3.1)

thanks,
Al