Subject: [bmwg] review of IPv6 Methodology draft

From: [email-masked] (Scott O. Bradner)
Date: Sun, 8 Apr 2007 17:00:11 -0400 (EDT)

Last Call Review Template

I-D Title(s): IPv6 Benchmarking Methodology
Filename(s): draft-ietf-bmwg-ipv6-meth-01.txt
Reviewer Name: Scott Bradner
Date: 4/9/07

Please organize your comments in the following categories below.

Review Summary:

Overall:

   * Does/Do the draft(s) provide clear identification of the
     scope of work? E.g., is the class of device, system, or
     service being characterized clearly articulated. 

	yes

   * If a terminology memo, are the measurement areas clearly
     defined or otherwise cited?  Is the working set of
     supporting terminology sufficient and correct?  To your
     knowledge, are the areas of the memo that may conflict
     with other bodies of work? Are there any measurements or
     terminology that are superfluous?  Are any missing?

	n/a

   * If a methodology memo, does the methodology AND its
     corresponding terminology adequately define a benchmarking
     solution for its application area? Do the methodologies present
     sufficient detail for the experimental control of the benchmarks?

	yes

   * If neither a terminology or methodology, does the offered
     memo offer complementary information important to the use
     or application of the related benchmarking solution?

	n/a

   * Do you feel there are undocumented limitations or caveats to
     the benchmarking solution being proposed?  If so, please
     describe.

	no

   * Does the memo attempt to define acceptance criteria for
     any of the benchmark areas?

	no

Technical Content:  (Accuracy, Completeness of coverage)

   Are definitions accurate? Is the terminology offered relevant?

	n/a

   To your knowledge, are there technical areas that are erroneous?
   Are there questionable technical areas that need to be re-examined
   or otherwise scrutinized.

	no

   Does the solution adequately address IPv6?

	yes

   Do you feel the memo(s) being offered are technically mature enough
   for advancement to informational RFC?

	yes

Clarity and Utility:

  If you had a need, would you utilize the benchmarking solutions
  advocated by this and its related memos?  If not, why?

	see suggestions at the end

Conformance to BMWG principles: (see BMWG charter)
http://www.ietf.cnri.reston.va.us/html.charters/bmwg-charter.html

  Do you have confidence that the benchmarks, as explicitly
  defined, will yield consistent results if repeated on the
  same device (DUT/SUT), multiple times for a given test condition.
  If not, cite benchmark(s) and issue(s).

	yes

  Do you have confidence that the benchmarks, if executed for a
  given test condition, utilizing the documented methodology
  on multiple test infrastructure (e.g., test equipment), would
  yield correct and consistent results on the same DUT/SUT?
  (Said differently, are the benchmark's methodology written
  with enough exacting detail, that benchmark implementation
  differences do not yield a difference in the measured quantities?)
  If not, cite benchmark(s) and issue(s).

	yes

  Do you feel that the benchmarks form a basis of comparison between
  implementations of quantity being characterized? (I.e., are the
  benchmarks suitable for comparing solutions from different vendors.)

	yes

  If not, cite benchmarks and issues.

  For those benchmarks cited above, do you feel that the benchmarks,
  as specified, have universal applicability for the given
  behavior being characterized?  (i.e., benchmarks might not form
  a basis for cross-vendor comparison, can be used universally
  in a different role.)

	yes

Editorial Comments:
(includes any deficiencies noted w.r.t. I-D Nits, spelling, & grammar)

section 4 - in "note" - change "The dynamic option is preferred if
the test tool interacts with the DUT" to ""The dynamic option is
preferred wherein the test tool interacts with the DUT"

section 5.1 - change "Two types of media are commonly deployed and
SHOULD be tested" to "Two types of media are commonly deployed and each
SHOULD be tested if the network element supports that type of media:"

section 5.1.1 - append "if supported" to "The 4096, 8192, 9216 bytes
long jumbo frame sizes SHOULD be used when benchmarking Gigabit Ethernet
interfaces." 

section 5.2.1 - add '[ note to IANA, replace "xxxxx" with assigned
prefix]' after "IANA reserved the IPv6 address block xxxxx/48 for use
with IPv6 benchmark testing."

change "These addresses MUST not be assumed to be routable on the
Internet" to "These addresses MUST be assumed to be not routable on the
Internet"

last pp in section 5.2.1 - it would be good to say why 

section 5.3 3rd pp - this is true as long as hop-by-hop header
processing impacts more than an interface processor - I would expect
that a future generation of v6 routers will implement this processing in
firmware (since its not hard to do in general and because it protects
the router against DoS attacks based on adding the header

6th pp - "bottom size"??

section 8 - I'd remove the sentence beginning "Most network
infrastructures are allocated a /48" because this is no longer the
standard assignment - just say that the requested size will meet the
requirements for testing in large routers and large emulated networks

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg