Subject: [bmwg] Draft draft-ietf-bmwg-mcastm-10.txt Last Call

From: "Perser, Jerry" <[email-masked]>
Date: Wed, 22 Jan 2003 15:13:04 -0800

It seems that we have been her before.  Once again I have concerns that the
"Methodology for IP Multicast Benchmarking" draft is not ready.  Some of
these go back to the 1999 meetings.  I would like to open this up for
rebuttal, not from just the authors, but the entire BMWG community:

1. Single ingress port.  Section 3 (end of first paragraph) state
"Methodologies for multiple ingress, multiple egress scenarios are beyond
the scope of this document."

Every single multicast test I have been asked to do has required either
multiple ingress or multiple egress ports.  I have yet run across someone
who wants to benchmark only two ports on a DUT/SUT.

The question I have is two part: A.) Does anyone has a real world experience
where you would ONLY benchmark two ports?  B.) Should this draft be expanded
to include the common scenarios, or another draft be written to cover these
scenarios?

2. Ambiguous.  If you go back in the archives, you will find that there have
been issues with the drafts ambiguity in sections 4.1, 8.1, and 8.2 since
1999.  Back in 1999, we had appendix A to give a hint of what to expect the
offered load to look like.  Without appendix A (or the algorithms used to
make appendix A), how can one predict what packets will be offered to the
DUT/SUT?

Since the packets are to be offered can not be predicted, how do we know
that two scripts provided by two vendors will give similar results on the
same DUT/SUT?  Since they both comply with the ambiguous methodology, both
results are right.  Can someone argue that a DUT/SUT will always give the
same throughput no matter how unicast and multicast packets are offered?

3. RFC2432 harmony.  In reading RFC2432, I found a few things
different/missing in the methodology draft.  In RFC2432, the Aggregated
Multicast Throughput talks about rate as a function of number of destination
ports.  While the methodology's procedure has a fixed offered load and
varies the ports as a function of OLOAD.  Which variable is fixed, the
number of ports or the offered load?

Also RFC2432 has sections labeled "Issues:".  I can not find where RFC 2432
section "3.2.3 Aggregated Multicast Throughput" Issues are addressed in the
draft.  They are not addressed in draft's section "4.3 Aggregated Multicast
Throughput".  Do the authors need to follow RFC2432?

4. Plagiarism. Most of the draft's objectives is a copy the RFC2432's
definitions.  Section 8 is a copy of RFC 2432 section 3.6 (minus the grammar
error I pointed out in 1999).  Section 8's paragraph serves no propose if
the reader reads and follows section 1's instructions (second paragraph).

While there is no technical issue here, I would like to raise a moral one.
My professors frown on submitting any work that was not original.
Submitting other people work was grounds for expulsion.  As educated
authors, how much of the previous work we should copy and label our own?

Jerry.

___________________________________________________________ 
Jerry Perser                         Spirent Communications
Director of Test Methodology         SmartBits Division 
Phone: 818.676.2320                  26750 Agoura Road 
Lab:   818.676.2337                  Calabasas, CA, 91302 
Cell:  818.292.6457                  Corp:  818.676.2300
_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg