Subject: [bmwg] WG Last Call: draft-ietf-bmwg-mcastm-08.txt

From: Kevin Dubray <[email-masked]>
Date: Thu, 08 Aug 2002 11:10:48 -0400

BMWG'ers:

A WG Last Call period for the Internet-Draft on
      "Methodology for IP Multicast Benchmarking,"

      <draft-ietf-bmwg-mcastm-08.txt>

will be open from 08 August 2002 until 29 August 2002.

This latest version of the I-D attempts to address feedback
from the previous WG Last Call.

If you feel that this draft should not be given to the Area Directors
for consideration in progressing the I-D to an Informational RFC,
please cite your basis to this list or [email-masked].  This must
be done prior to close of the last call period. Of course, notes of
support are always welcome.

A URL for the Internet-Draft is:

http://www.ietf.org/internet-drafts/draft-ietf-bmwg-mcastm-08.txt

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: "David Newman" <[email-masked]>
Date: Wed, 28 Aug 2002 18:52:30 -0700

Some minor comments:

1. An objection was voiced about version 7 that the text for the test
objtectives in section 4 simply restated definitions from RFC 2432. This
objection has not been addressed in version 8.

For example, in section 4.2 on SGFM, the objective is " A table that
demonstrates Forwarding Rate as a function of tested multicast groups for a
fixed number of tested DUT/SUT ports."

That's a definition, not an objective. Further, it was already done in 2432.

These is easy to fix -- just turn it around to say (for example) "to
determine forwarding rate as a function of tested multicast groups..."

2. Section 5 on latency says this methodology extends the latency
measurements in RFC 2544, but in some ways it redefines "latency" rather
than extending the term. Consider the following text:

  In order to lessen the effect of packet buffering in the DUT/SUT,
     the latency tests MUST be run such that the offered load is less
     than the multicast throughput of the DUT/SUT as determined in the
     previous section.

If I understand this correctly, the offered load here would be less than the
throughput level. This is at variance with RFC 2544, section 26.2, which
mandates that latency measurements be taken *at* the throughput level.

There have been discussions on this list about latency and load, with some
suggesting we do things differently than what is given in 2544.

To maintain the spirit of extending 2544, I would recommend this document
measure latency at the throughput level, but also allow other iterations at
different load levels.

3. I apologize for only now raising this on last call, but I only recently
came across the issue in testing. Neither the methodology nor the RFC 2432
terminology document describe metrics for mixed-class latency or jitter.

There is definitely value in such metrics, and I would recommend they be
added.

Again, sorry I didn't see this before now.

dn

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of Kevin
> Dubray
> Sent: Thursday, August 08, 2002 8:11 AM
> To: [email-masked]
> Subject: [bmwg] WG Last Call: draft-ietf-bmwg-mcastm-08.txt
>
>
> BMWG'ers:
>

---

From: "David Newman" <[email-masked]>
Date: Wed, 28 Aug 2002 18:52:30 -0700

Some minor comments:

dn

---

From: Kevin Dubray <[email-masked]>
Date: Thu, 29 Aug 2002 17:53:14 -0400

> measure latency at the throughput level, but also allow other iterations at

So, you're saying wording like "Multicast Latency MUST be collected
at the measured MCT level of the DUT; multicast latency at other
offered loads is OPTIONAL." should be added.  I think that's fine.

> There is definitely value in such metrics, and I would recommend they be

I agree that there might be benefit to such measurements. But let's
get this I-D as a RFC out before we increase scope. ;-)  We can
rework the RFC or issue a supplement if there's interest or need.

-Kevin

---

From: Kevin Dubray <[email-masked]>
Date: Thu, 29 Aug 2002 17:53:14 -0400

-Kevin

---

From: "David Newman" <[email-masked]>
Date: Thu, 29 Aug 2002 15:14:08 -0700

> offered loads is OPTIONAL." should be added.  I think that's fine.

Yes, that's it.

In section 5, I would also recommend clarifying or deleting text about
reducing the effects of buffering. While that is a commendable goal, I do
not believe there is a universal backoff factor that works for every test
case on every DUT/SUT. The text Kevin suggests above would leave that up to
the user.

John Dawson started a interesting thread on this list some time back about
latency vs. load, but getting into that here would probably expand the scope
of the document beyond what the authors (or the WG chair) intended. All the
same, those issues are definitely relevant to this part of the methodology.

dn

---

From: "David Newman" <[email-masked]>
Date: Thu, 29 Aug 2002 15:14:08 -0700

Yes, that's it.

dn

---

From: Kevin Dubray <[email-masked]>
Date: Thu, 29 Aug 2002 18:37:24 -0400

Some other input gleaned from others:

Dave Meyer points out section 3.1.2 on addressing is out of date
relative to current IANA assignments. (Thanks, too, to Zaid Albanna
for giving some history in the migration of the multicast test
addresses.)  Dave and I work on the following alternative
wording:

"It is intended that the collection of benchmarks prescribed in
this document be executed in an isolated lab environment.  That
is to say, the test traffic offered the tested devices MUST NOT
traverse a live internet, intranet, or other user-oriented network.

Assuming the above, there is no restriction to the use of multicast
addresses to compose the test traffic other than those assignments 
imposed by IANA.  The IANA assignments MUST be regarded for
operational consistency.  For multicast address assignments see:

      http://www.iana.org/assignments/multicast-addresses

It should be noted that address selection need not be restricted to
Administratively Scoped IP Multicast addresses."

Also, I think wording suggested by Mr. Perser in an earlier
email should be incorporated as a general, required procedural
step for forwarding metrics:

[Test device] "ports MAY transmit some learning frames for the
DUT/SUT to populate its forwarding tables."

I'd change this to a MUST as some scenarios need to prime the SUT as
a function of RPF or device's caching mechanisms.

Lastly, when RFC 2432 was put out, DVMRP was the driver for
the encapsulation-related metrics.  It appears that folks
employ other tunneling mechanisms (e.g., GRE, MPLS) to carry
their multicast through the unicast world.  Do folks agree
that the level of detail stated in sections 4.4.1 - 4.4.3 is
adequate?

-Kevin

---

From: Kevin Dubray <[email-masked]>
Date: Thu, 29 Aug 2002 18:37:24 -0400

-Kevin