Subject: [bmwg] Review of IPv6 Benchmarking Methodologies draft

From: "Bill Cerveny" <[email-masked]>
Date: Thu, 15 Mar 2007 10:58:04 -0400

Below is my review of the IPv6 Benchmarking Methodologies draft for its
WGLC using the last call review template.  I
hadn't reviewed previous comments/reviews on the list, so it is possible
I've duplicated earlier comments.

Bill Cerveny

Last Call Review Template

I-D Title(s): IPv6 Benchmarking Methodology

Filename(s):draft-ietf-bmwg-ipv6-meth-01.txt
Reviewer Name:
Date: March 14, 2007

Please organize your comments in the following categories below.

Review Summary:
This document accurately captures the current state-of-the-art for 
performance issues that should be tested in the IPv6-enabled devices.  

Overall:

   * Does/Do the draft(s) provide clear identification of the
     scope of work? E.g., is the class of device, system, or
     service being characterized clearly articulated.

Yes.

   * If a terminology memo, are the measurement areas clearly
     defined or otherwise cited?  Is the working set of
     supporting terminology sufficient and correct?  To your
     knowledge, are the areas of the memo that may conflict
     with other bodies of work? Are there any measurements or
     terminology that are superfluous?  Are any missing?

   * If a methodology memo, does the methodology AND its
     corresponding terminology adequately define a benchmarking
     solution for its application area? Do the methodologies present
     sufficient detail for the experimental control of the benchmarks?

Yes, excluding the areas discussed in the editorial comments. In
particular, some
clarification in the area of filtering could be helpful.

   * If neither a terminology or methodology, does the offered
     memo offer complementary information important to the use
     or application of the related benchmarking solution?

   * Do you feel there are undocumented limitations or caveats to
     the benchmarking solution being proposed?  If so, please
     describe.

No

   * Does the memo attempt to define acceptance criteria for
     any of the benchmark areas?

Technical Content:  (Accuracy, Completeness of coverage)

   Are definitions accurate? Is the terminology offered relevant?

Yes and yes

   To your knowledge, are there technical areas that are erroneous?
   Are there questionable technical areas that need to be re-examined
   or otherwise scrutinized.

Not to my knowledge

   Does the solution adequately address IPv6?

Yes

   Do you feel the memo(s) being offered are technically mature enough
   for advancement to informational RFC?

Yes.  This said, the amount of operational experience with IPv6 has been 
very limited, compared to that of IPv4.  It is likely that the document 
will need to be updated to reflect new findings as
more is understood with IPv6 operationally.  On the other hand, this 
document will provide valuable guidance to early adopters.

Clarity and Utility:

  If you had a need, would you utilize the benchmarking solutions
  advocated by this and its related memos?  If not, why?

Yes, they are consistent will what I would consider if I had a need to 
perform general IPv6 benchmark and performance testing.

Conformance to BMWG principles: (see BMWG charter)
http://www.ietf.cnri.reston.va.us/html.charters/bmwg-charter.html

  Do you have confidence that the benchmarks, as explicitly
  defined, will yield consistent results if repeated on the
  same device (DUT/SUT), multiple times for a given test condition.
  If not, cite benchmark(s) and issue(s).

Yes

  Do you have confidence that the benchmarks, if executed for a
  given test condition, utilizing the documented methodology
  on multiple test infrastructure (e.g., test equipment), would
  yield correct and consistent results on the same DUT/SUT?
  (Said differently, are the benchmark's methodology written
  with enough exacting detail, that benchmark implementation
  differences do not yield a difference in the measured quantities?)
  If not, cite benchmark(s) and issue(s).

Yes

  Do you feel that the benchmarks form a basis of comparison between
  implementations of quantity being characterized? (I.e., are the
  benchmarks suitable for comparing solutions from different vendors.)

Yes

  If not, cite benchmarks and issues.

  For those benchmarks cited above, do you feel that the benchmarks,
  as specified, have universal applicability for the given
  behavior being characterized?  (i.e., benchmarks might not form
  a basis for cross-vendor comparison, can be used universally
  in a different role.)

Yes

Editorial Comments:
(includes any deficiencies noted w.r.t. I-D Nits, spelling, & grammar)

Review of draft-ietf-bmwg-ipv6-meth-01.txt

Section 1. 
s/are proving to be very useful/are proving to be useful/

s/procedures as described in RFC2544 and not to replace them/procedures 
described in RFC2544 and not replace them/

Section 4. 

s/test traffic simulated end points/test traffic simulates end points/

Sentence beginning "The test scenarios assume ..." Consider 
rewording to something like: "To avoid neighbor solicitation (NS) and
neighbor advertisement (NA) storms due to the neighbor unreachability
detection
(NUD) mechanism, the test scenarios assume the test traffic simulates
end
points and the IPv6 source and destination addresses are one hop beyond
the DUT."

Section 5

Change "Also, not all network elements support this type of addresses"
to something like "Also, not all network elements support addresses of
this
prefix length."

s/Interface ID portion of the global/interface ID portion of global/

s/tests be conducted using the following lengths/tests be conducted
using the following prefix lengths/

Consider changing sentence beginning "Other prefix lengths can also 
be used..." to "Other prefix lengths can be used. However, the indicated
range should be sufficient to establish baseline performance metrics."

Regarding prefix lengths and IANA recommendation -- To be consistent
with recommendations in document, the IANA recommendations should be for
a prefix /31 or shorter.  If a
shorter IANA prefix is obtained, either all references recommending
prefixes shorter
than the IANA block should be adjusted and/or there should be text
indicating
that some tests may use blocks other than the IANA block.

There is no section 4.2.1 referencing IANA recommendations

Uncapitalize neighbor discovery, neighbor advertisement, neighbor
solicitation, neighbor unreachability, hop-by-hop, source addresses,
destination addresses, protocol, address
plans, preamble, inter frame gap.

s/various types of practical traffic such as: Fragmented/various types
of practical traffic such as fragmented/

Consider a scenario where specific types of extension headers either
are blocked or not used on the network of the customer paying for the
test -- is it still considered
necessary to test with these headers?

s/Considering the fact that/Considering that/

s/containing this extension headers type/containing this extension
header type/

s/extension headers processing capability which/extension header
processing capability, which/

s/The tests with traffic containing each individual extension header
MUST be complemented with tests that contain/The tests with traffic
containing each individual extension
header MUST be complemented with tests containing/

ESP not defined earlier, as far as I can tell.

s/The extension headers chain/The extension header chain/

s/real life extension headers chain/real life extension header chain/

s/the extension headers chain SHOULD/the extension header chain SHOULD/

s/For the most cases/For most cases/

s/it is most likely/it is likely/

Section 6

SA and DA defined twice.

Perhaps RFC2544 explains this better, but shouldn't the addresses in the
filter examples have prefix lengths, such as 2001:DB8::1/40?

In the text "The protocol field is defined as ...", shouldn't it be
indicated that this is not intended to be an all-inclusive list of
protocols??

Section 7

s/Extension headers specific/Extension header specific/

s/For these reasons, this test is not recommended anymore for IPv6
benchmarking/For these reasons, this test is no longer recommended for
IPv6 benchmarking/

Section 8

As mentioned earlier an IANA /48 wouldn't be sufficient, as the tests in
this document are currently described.

Appendix A

s/4 bytes header/4-byte header/

s/a 2 or 4 bytes FCS field and a 1 byte/a 2- or 4-byte FCS field and a
1-byte/

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg