Subject: [bmwg] I-D ACTION:draft-ietf-bmwg-mcastm-11.txt

From: [email-masked]
Date: Thu, 06 Mar 2003 06:24:26 -0500

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology Working Group of the IETF.

	Title		: Methodology for IP Multicast Benchmarking
	Author(s)	: D. Stopp, B. Hickman
	Filename	: draft-ietf-bmwg-mcastm-11.txt
	Pages		: 28
	Date		: 2003-3-5
	
The purpose of this draft is to describe methodology specific to 
the benchmarking of multicast IP forwarding devices. It builds upon 
the tenets set forth in RFC 2544, RFC 2432 and other IETF 
Benchmarking Methodology Working Group (BMWG) efforts.  This 
document seeks to extend these efforts to the multicast paradigm. 
The BMWG produces two major classes of documents: Benchmarking 
Terminology documents and Benchmarking Methodology documents. The 
Terminology documents present the benchmarks and other related 
terms. The Methodology documents define the procedures required to 
collect the benchmarks cited in the corresponding Terminology 
documents.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-bmwg-mcastm-11.txt

To remove yourself from the IETF Announcement list, send a message to 
ietf-announce-request with the word unsubscribe in the body of the message.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-bmwg-mcastm-11.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-bmwg-mcastm-11.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

---

From: Debby Stopp <[email-masked]>
Date: Thu, 20 Mar 2003 12:10:47 -0800

Let's stop using the terms LIFO and FIFO because these terms are used to
define how entities arriving at a queue get serviced.  If you disagree, then
I would suggest you review the enclosed links:  The National Institute of
Standards and Technology (http://www.nist.gov/dads/HTML/lastfirstout.html)
and the General Services Administration
http://glossary.its.bldrdoc.gov/fs-1037/dir-020/_2990.htm for the
appropriate use of these definitions.  Unless, of course, you wish to
redefine these terms for meauring delay between two separate events.  In
that case, a terminology document will be required.

Now that we cease to use this terms inappropriately, let's get to your
perceived issue.  You believe that the authors have introduced the LIFO
goalposts from RFC 1242.  I disagree.  I don't see any explicit references
to RFC 1242 in section 6.1 or 6.2.  The group join delay measures the time
elapsed between two different externally observable events.  It is
irrelevent for this test if the DUT is a "store and forward" or a "bit
forwarding" device.  

	According to Section 6.1, the Group Join delay time is defined as
the difference between two timestamps A and B where:

	Timestamp A MUST be the time the last bit of the IGMP group
membership report is sent from the destination test port
	timestamp B MUST be the time the first bit of the first valid
multicast frame is forwarded on the egress interface of the DUT/SUT. 

The packets sent to and received from the egress interface are different.
One packet is an IP packet with an IGMP message.  This packet is not being
forwarded on the data plane but rather being processed by the state machine.
The other packet is an IP or UDP packet with a destination multicast address
that has been replicated and forwarded on the data plane once the join is
complete. Please note that any inferences to the LIFO are your preceived
notions.  

I am sure that there are differing opinions on where the goalpost should be
placed.  We elected to start the clock when the last bit of the IGMP message
is placed on the wire in order to exclude the time to transmit IGMP packet;
we elected to stop the clock on receipt of first bit of the forwarded
multicast packet in order to exclude the size of the validatation packet as
part of the metric.  Regardless of the goalpost placement, the objective of
the methodology is to create a meaningful and repeatable  metric.  I believe
we meet that objective.

Debby Stopp

-----Original Message-----
From: David Newman [mailto:[email-masked]] 
Sent: Tuesday, March 18, 2003 10:59 AM
To: [email-masked]
Subject: RE: [bmwg] I-D ACTION:draft-ietf-bmwg-mcastm-11.txt

Section 1.

"This document defines tests for measuring and reporting the forwarding,
latency and IGMP group membership characteristics of  devices that support
IP multicast routing protocols."

s/routing //

This document covers forwarding, not routing. This is consistent with
"forwarding" in the abstract of 2432.

Section 3.1.4

"The TTL in IGMP control plane messages is in compliance with the version of
IGMP in use."

s/is/MUST be/

Section 6.1

Timestamp A MUST be the time the last bit of the IGMP group membership
report is sent from the destination test port; timestamp B MUST be the time
the first bit of the first valid multicast frame is forwarded on the egress
interface of the DUT/SUT.

s/first bit/last bit/

Add at end:

IGMP group membership MUST be verified through externally observable means.

The LIFO method given here will not produce meaningful numbers for
store-and-forward devices. Two reasons:

1. Store-and-forward devices need the entire packet before processing begins
2. It's impossible to validate group membership with the "FO" in LIFO.

On point 1, some of the authors have objected that LIFO vs. FIFO is
irrelevant since this is a test of delay and not latency as defined in 1242
and 2432.

This is a red herring: While WHAT we measure has changed, HOW we measure it
has not. The authors introduced the LIFO goalposts from 1242; they cannot
then claim that LIFO is irrelevant.

IOW, the fact that we measure a two-way conversation multiple packets doesn'
t remove the requirement for meaningful starting and ending points. RFC 2432
doesn't define these, and the ending point with the FO method given here isn
't meaningful. This brings us to point 2, the lack of a validation
requirement.

Simple analogy illustrates the validation issue. I have a friend who is a
firearms collector. He sends away for a subscription to Guns & Ammo
magazine. My friend then receives a magazine in a manila envelope. He rips
open the envelope only to discover he has received Gun Control Gazette
(something I'm more likely to subscribe to).

The "FO" in LIFO is the equivalent of my friend receiving the envelope. A
LILO method would allow my friend to ascertain that his join request has
failed.

Note that LILO itself doesn't guarantee a valid join; a separate step of
externally observable verification is needed for that. Still, it's not
possible to do this separate step with LIFO.

Section 6.2 on group leave gets it right: LILO.

dn

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of 
> [email-masked]
> Sent: Thursday, March 06, 2003 3:24 AM
> To: IETF-Announce:
> Cc: [email-masked]
> Subject: [bmwg] I-D ACTION:draft-ietf-bmwg-mcastm-11.txt
>
>
> A New Internet-Draft is available from the on-line Internet-Drafts 
> directories. This draft is a work item of the Benchmarking Methodology 
> Working Group of the IETF.
>
> The purpose of this draft is to describe methodology specific to the 
> benchmarking of multicast IP forwarding devices. It builds upon the 
> tenets set forth in RFC 2544, RFC 2432 and other IETF Benchmarking 
> Methodology Working Group (BMWG) efforts.  This document seeks to 
> extend these efforts to the multicast paradigm. The BMWG produces two 
> major classes of documents: Benchmarking Terminology documents and 
> Benchmarking Methodology documents. The Terminology documents present 
> the benchmarks and other related terms. The Methodology documents 
> define the procedures required to collect the benchmarks cited in the 
> corresponding Terminology documents.
>
> Internet-Drafts are also available by anonymous FTP. Login with the 
> username "anonymous" and a password of your e-mail address. After 
> logging in, type "cd internet-drafts" and then
> http://www.ietf.org/shadow.html or 
> ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>
>

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg
_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg