Subject: [bmwg] Comments to draft-ietf-bmwg-acc-bench-term-09.txt

From: "Chip Popoviciu \(cpopovic\)" <[email-masked]>
Date: Sun, 6 Aug 2006 16:12:28 -0400

Scott and Shankar,
 
Here are a few comments related to draft-ietf-bmwg-acc-bench-term-09.txt:
- Page 3, what does "can be benchmarked with faster test duration" mean? Was it meant "can be benchmarked over shorter test durations"?
- Page 3, unclear "These benchmarks White Box benchmarks are provided in Appendix 1"
- Page 4, table 1, Not sure why the Instability Phase is marked by "Achieve Configuration Set". Is that DUT configuration, instability conditions configuration?
- Page 5 and others: Why are the Measurements Units N/A?
- Page 6, sections 3.1.3 Startup Conditions: Why not management plane as well? At startup we might have stress generated by provisioning mechanisms.
- Page 6, Aggregate Forwarding Rate, should we mention that this is unidirectional?
- Page 15, Management Plane, do we consider any provisioning functions that the router might have to do? i.e., it managing other devices?
- Page 15, Security Plane, User Access via tunnels is considered security plane? This is because we refer to IPsec tunnels? Setting up other types of tunnels couldn't fit in the Control or Management Plane? I think the Security Plane needs a more specific definition.
- Page 17, section 3.4.2.2 why not use percentage degradation?
- Page 18, section 3.4.2.3. This formula is unclear, it seems to indicate that from the sum of all stable samples we subtract the degraded samples and then divide by the number of samples. This approach would assume that the pattern observed while taking the samples during startup is relevant during instability, i.e. there is a correlation between sample n from startup and sample n from instability measurement. I don't think such correlations could be identified so the formula should rather be presented as a difference between averages. This comment applies to all other formula in this section. If one would like to observe a true pattern for degradation, then the degraded measurements should be subtracted from an average stable aggregate.
- Page 20, would, large amounts of SNMP polls lead to instability of the Management Plane, or provisioning requests
 
Editorial:
- Abstract: replace "configuration terms associated with the each phase." with "configuration terms associated with each phase."
- Page 3: replace "associated with the each phase" with "associated with each phase"
- Page 7: replace "The test equipment is also to monitor" with "The test equipment is also capable to monitor"
- Page 11: replace "Stable Aggregate Forwarding Rate so that it measured" with "Stable Aggregate Forwarding Rate so that it is measured"
 
Best Regards,
Chip
_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg