Subject: BMWG Minutes of 47th IETF

From: Kevin Dubray <[email-masked]>
Date: Wed, 19 Apr 2000 18:38:02 -0400

Below are the unedited minutes from the BMWG session
at the Adelaide IETF.

-------------------------------------------------------------------

Benchmarking Methodology WG Minutes

WG Chair: Kevin Dubray

Minutes reported by Cynthia Martin.

1. Administrative Overview.

Kevin Dubray provided an overview of the meeting agenda.  No questions 
or comments were given from the audience.

2. Overview of ID "ATM ABR Benchmarking Terminology".

Jeff Dunn provided an overview of the "ATM ABR Benchmarking Terminology" 
ID including updates and changes to the ID from previous versions.  There is 
interest in the group to create an "ATM ABR Benchmarking Methodology" ID, but 
no proposed dates or timelines were discussed.  Jeff requested that anyone 
interested in creating the methodology ID should contact Jeff at 
[email-masked] or via the BMWG mailing list.  We need people 
with operational experience to provide input.  The "ATM ABR Benchmarking 
Terminology" ID will go into last call after this BMWG meeting if no comments or 
questions are raised.

3. Overview of ID "Frame Relay Benchmarking Terminology".

Jeff Dunn provided an overview of the "Frame Relay Benchmarking Terminology" 
ID including updates and changes to the ID from previous versions.  The 
comment was made to add additional PDH (DS-3) and ADSL information in this 
ID or to create a new ID.    This issue will be put on the list so everyone 
may comment on what they feel is the appropriate solution.  The authors 
believe that a separate ID should be created for ADSL. The "Frame Relay 
Benchmarking Terminology" ID will go into last call after this BMWG meeting 
if no comments or questions are raised.

4. Overview of ID "ATM Benchmarking Methodology".

Jeff Dunn provided an overview of the "ATM Benchmarking Methodology " ID 
including updates and changes to the ID from previous versions.  Many comments 
and questions were raised including:

  a) Should the required histograms be normalized or absolute?
  After a discussion, it was decided that both should be available to the 
  user and that a uniform format is more important.  The ID will reflect this 
  decision.

  b) How do we test cell rate margin?  
  The Cell Rate Margin is defined the terminology ID, but we have not come 
  up with a good test scenario for this.  The authors would like to remove 
  this test from the ID.

  c) The filter definitions are still not defined.  Should we include them 
  in the ID?  Are the filters for ATM or IP?  Does it effect throughput?  These 
  questions will be put on the list so everyone may comment.  We need an answer 
  to these for the final version of the ID.  The authors would like to remove 
  this from the ID.

The "ATM Benchmarking Methodology" ID will go under one more revision before 
the next IETF and then into last call.

5. Overview of IP QOS ID proposal

Jerry Perser provided an overview of the "IP QOS" ID.  Many comments and 
questions were raised including:

  a) Many people are interested in the ID, but who is willing to work and 
  comment on it?
  b) IP QOS is not defined or understood, how will this ID be developed?
  c) Should we address DIFFSERV?   How does this interact with IP QOS?  DIFF
SERV 
  does not define specific metrics or yardsticks for service agreements.  How 
  will this effect the development of the ID?
  d) Should we address RSVP? How does this interact with IP QOS?
  e) Should we address TOS? How does this interact with IP QOS?
  f) Should this be a configuration effort, referring to other work and re-
  interpreting existing metrics?
  g) Should we also address IP over ATM QOS and IP over Frame Relay QOS to 
  narrow the scope?
  h) Is this tied to certain mechanisms and algorithms?
  i) Is this ingress traffic policing or something else on the egress?  Is 
  this traffic policing conformance?

The attendees agreed that the goals of the proposal need to be reworked and
articulated to the BMWG 
mailing list.

6. Overview of Routing Metrics proposals

  a) Guy Trotter provided an overview of the proposed "Forwarding Performance 
  based on Forwarding Table Size" ID.  A few questions were raised including:
    i. Will the ID address forwarding table or route table?
    The author clarified that the ID will address forwarding table only, the 
    nomenclature needs to be consistent.
    ii. Will the ID address issues like how soon after the routing distribution 
    is complete will traffic forwarding occur?
    The ID needs to address this.
    iii. Will the ID address the effects of how the routing entries are placed 
    into the table and if it effects traffic forwarding?  What are the 
    boundaries on the ID?  The ID needs to address this.

  b) Jeff Dunn provided an overview of the proposed "Framework for IPv4 
  Routing Benchmarking" ID and "IPv4 Routing Metrics" ID.  A few questions 
  	were raised including:
     i. Will the ID address MPLS or QOS?
     The author clarified that the ID will not address QOS that is for the QOS 
     ID discussed earlier.  The ID will not address MPLS at this time.  This 
     will be put on the list so everyone may comment.  The authors believe that 
     MPLS should be discussed in another ID, similar to the ATM and Frame Relay 
     ID's.
     ii. What is the scope of the ID?
     The ID needs to be very clear.  With more people working in the ID, the 
     authors believe that a clearly defined, well-scoped ID can be created.
     iii. Does this proposed ID overlap with the proposed "Forwarding 
     Performance based on Forwarding Table Size" ID?
     Yes.  The proposed "Forwarding Performance based on Forwarding Table 
     Size" is dependent on the proposed "Framework for IPv4 Routing 
     Benchmarking", but they can be developed in parallel.  The 
     framework is necessary to develop all other router based benchmarking 
     ID's.

     c) Although a consensus was not taken, the group agreed that both 
     proposals could be developed at the same time.  Since the BMWG meeting, 
     the authors of both ID's decided to work together on the "Framework for 
     IPv4 Routing Benchmarking" ID.  A first draft of this ID should be 
     available in 1-2 months from the BMWG meeting date. A first draft of 
     the proposed "Forwarding Performance based on Forwarding Table Size" ID 
     was not discussed, but after the meeting the author did indicate that a 
     first draft would be available within a few weeks from the BMWG meeting 
     date.