Subject: [bmwg] Updated draft on MPLS forwarding Benchmarking: draft-akhter-bmwg-mpls-meth-03.txt

From: "Aamer Akhter (aakhter)" <[email-masked]>
Date: Mon, 26 Nov 2007 12:05:52 -0500

Hello BMWG,

We would like to bring to your attention the updated draft on basic MPLS forwarding that we will be submitting to the IETF as an individual submission (once submissions opens up again). There have been significant changes since the last version of the draft including:

* removal of many test cases thought to be out of scope
* rewriting of many cases to be less verbose
* addition of reload tests

http://www.employees.org/~aamer/draft-akhter-bmwg-mpls-meth-03.txt

   Over the past several years MPLS networks have gained greater 
   popularity. However, there is no standard method to compare and 
   contrast the varying implementations and their strong and weak 
   points. This document proposes a methodology using common criteria 
   for the comparison of various implementations of basic MPLS 
   forwarding devices.

If there are any concerns or changes that need to be made please do not hesitate to contact us.

Regards,

Aamer Akhter
Rajiv Asati

==================

	Title		: MPLS Benchmarking Methodology
	Author(s)	: Aamer Akhter and Rajiv Asati
	Filename	: draft-akhter-bmwg-mpls-meth-03.txt
	Pages		: 21
	Date		: 2007-11-25
	
   The purpose of this draft is to describe a methodology specific to 
   the benchmarking of MPLS forwarding devices. The scope of this 
   benchmarking will be limited to various types of packet-forwarding 
   and delay measurements. It builds upon the tenets set forth in 
   RFC2544 [RFC2544], RFC1242 [RFC1242] and other IETF Benchmarking 
 
   The BMWG produces two major classes of documents: Benchmarking 
   Terminology documents and Benchmarking Methodology documents.  The 
   Terminology documents present the benchmarks and other related 
   terms. The Methodology documents define the procedures required to 
   collect the benchmarks cited in the corresponding Terminology 
   documents.

A temporary URL for this Internet-Draft is:
http://www.employees.org/~aamer/draft-akhter-bmwg-mpls-meth-03.txt

-- 
Aamer Akhter / [email-masked]
Rajiv Asati / [email-masked]

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: "Rajiv Papneja" <[email-masked]>
Date: Thu, 29 Nov 2007 14:11:49 -0500

Hello Aamer, Rajiv

Thanks for sending the pointer to your dradt for review. Considering that
there is an ongoing effort on LDP, and RSVP-TE convergence benchmarking this
work complements the two efforts. I do support this work, however I do have
a couple of minor comments: 

In the section 6, under the test setup you recommend for the test cases
involving 2 or more labels, the tester should emulate MPLS services such as
L3VPN, or L2VPN to ensure that two label stack is imposed before the frame
is offered to the DUT. IMHO, one may create traffic streams with multiple
labels rather than intorducing the complexity of L3VPN/L2VPN configurations
and deviating from the true forwarding benchmarking style testing which this
draft proposes. Similar to IP forwarding benchmarking using static routes
rather than enabling routing protocols on the DUT and tester for throughput
measurements.

For section 6.4 and 6.5 since there are control plane recovery is also
important for correct traffic forwarding, the procedure should include
verifying whether the MPLS sessions are started to ensure new label bindings
are exchanged rather than tester using stale entries to send the traffic
streams.

Regards,
Rajiv

> -----Original Message-----
> From: Aamer Akhter (aakhter) [mailto:[email-masked]] 
> Sent: Monday, November 26, 2007 12:06 PM
> To: [email-masked]
> Subject: [bmwg] Updated draft on MPLS forwarding 
> Benchmarking:draft-akhter-bmwg-mpls-meth-03.txt
> 
> Hello BMWG,
> 
> We would like to bring to your attention the updated draft on 
> basic MPLS forwarding that we will be submitting to the IETF 
> as an individual submission (once submissions opens up 
> again). There have been significant changes since the last 
> version of the draft including:
>

---

From: "Silvija Andrijic Dry (sdry)" <[email-masked]>
Date: Fri, 30 Nov 2007 16:13:42 -0500

Hi Aamer, Rajiv,

I have read latest 03 rev as well some of your earlier ones and like
some of changes you have made especially adding system recovery and
reset test cases. Overall I think this is useful work and I am in
support of it.

Here are couple of specific comments:
- Section 2 (Scope): need to explicitly state that scope includes
unicast only and multicast is not within the scope (or if it is
additional considerations and test cases would definitely be needed).
Also I noticed IPv6 was mentioned in some other places in document (like
section 4.1.2 and 5) so would be good to be clear up front in scope
section what is covered by the draft 
- In section 6 (test procedure) you use as criteria to move to new test
iteration "if any frame loss is detected", while in section 4.1.5.1 you
say that "all traffic MUST be acocunted for, whether it was received on
wrong port, correct port or not received at all". I would make criterias
in these two instances consistant, preferably either frame loss or
receiption at the wrong port would require one to proceed to new test
iteration. Perhaps you can say in section 6 something like: "If traffic
verification procedure specified in 4.1.5.1 fails proceed to new
iteration" 
- In section 6.5 it would be good to suggest couple of more common
failure events besides device and software resets. For example
switchover between any redundant components in the DUT if any supported
(e.g. main route processor, switching fabric, etc.). Look at section 7
of MVPN Scale benchamrking draft where we make some additional common
failure events recommendations and some might be applicable in your
draft as well: 
http://www.ietf.org/internet-drafts/draft-sdry-bmwg-mvpnscale-03.txt 
- Also in section 6.5 it would be good to be more specific on what
device and software resets mean as you can typically reset device in
couple of different ways where not all result in same code paths and
outage times...I think the safest method to ensure consistancy would be
power cycle for device reset.

Cheers,
Silvija

-----Original Message-----
From: Aamer Akhter (aakhter) 
Sent: Monday, November 26, 2007 12:06 PM
To: [email-masked]
Subject: [bmwg] Updated draft on MPLS forwarding

Hello BMWG,

We would like to bring to your attention the updated draft on basic MPLS
forwarding that we will be submitting to the IETF as an individual
submission (once submissions opens up again). There have been
significant changes since the last version of the draft including:

If there are any concerns or changes that need to be made please do not
hesitate to contact us.

Regards,

Aamer Akhter
Rajiv Asati

==================