Subject: RE: [bmwg] Authors Action Items to Comments for draft-ietf-bmwg-i gp-datapla ne-conv-***-02

From: [email-masked]
Date: Wed, 9 Jun 2004 20:07:00 -0400

Robert,

Please find the following responses to additional comments you had made.
Follow-up comments from other BMWG participants is also welcome.

Scott
########################

1. In several sections the convergence state is
defined by the "forwarding rate" equal to the
"offered load". May be better to clarify that
the offered load must be less than or equal to the
capacity of the device.

RESPONSE - Forwarding Rate is an IETF (BMWG) defined term for an interface.

2.  Section 3.3 Route Convergence
This term seems misleading. The definition
implies the convergence of the routing function
on a single device. But "route convergence" implies
a source-destination path (e.g. multiple devices).

RESPONSE - The intention of this work item is to define terms and
methodologies for benchmarking convergence for a single routing device.  The
terms 'Route Convergence' and 'Network Convergence' are defined within this
work item.

3. Section 3.4 Full Convergence
The term "converge an entire route table" to
me implies completion of an algorithm.

RESPONSE - See the Applicability document for discussion of factors
contributing to convergence.

4.  Section 3.5 Convergence Packet Loss
The units in this and other sections are
measured in absolute packet counts. Would it
be feasible to consider percent of offered load?

RESPONSE - I do not think so, but if there is a justifiable technical reason
to use percentage of line rate then we will.

5.  Section 3.7 Convergence Recovery Instant
This section contains a hard constant.
This should be changed to a "provisional" or
suggested value.

RESPONSE - At the Minneapolis meeting it was agreed that a hard constant was
needed to make this benchmark consistent and repeatable.  5 seconds was the
suggested time to use.

6. Section 3.9 Convergence Event Transition
Is it possible that the forwarding rate may
never reach zero? In that case would not a
percentage of load or slope be more appropriate?

RESPONSE - This is definitely possible and will be considered in the next
revision.  Whether measured in pps or %, the non-zero minimum forwarding
rate must be addressed.

7. Section 3.9 Convergence Event Transition
It seems that this term might better be called
"Degradation Transition" to imply the downward
slope of the forwarding rate.

RESPONSE - I prefer the curent term, however if others see a benefit to
change it then I will.

8.  Section 3.10 Convergence Recovery Transition
For both this section and 3.9, it may be
more flexible to define this as a slope and a
percentage of forwarding rate. In that case,
the "threshold" crossed to determine state
could be manipulated. For example, instead of
defining this time as an absolute 100% recovery
of the forwarding state, it might be useful in some cases
to define recovery as 95% recovery of the forwarding state.

RESPONSE - Again, I prefer pps over %, but if there is a justifiable
technical reason to use percentage of line rate then we will.  I am opposed
to changing the definition of convergence to 95% (currently 100%) because of
the fact that the device has not achieved Full Convergence if 5% of the
route table does not converge.

9. Section 3.11 Loss-Derived Convergence Time
the phrase "converging many routes simultaneously".
I assume you mean processing multiple update messages?

RESPONSE - All terms are independent of the number of update messages.

10. Section 3.13 Restoration Convergence Time
Why is this a special case. It seems that this
would be the same as the other convergence measurements
but in this case, we would be transitioning to a previously
obtained (assumed desirable steady state).

RESPONSE - There are devices in which this has been exhibited to be a
special case.  BWMG participants from 2 service providers specifically
requested this be added to the 02 revision, as it was not in the 00 or 01
revisions.

11.  Additional flexibility such as using rates, slopes, and thresholds
would allow for
modification to the requirements for a device to "pass" the benchmark. 

RESPONSE - The BMWG is not in the business of defining criteria to PASS.

12. There should be "averages" defined so that a test can be conducted
     several times and the average or mean results can be recorded.

RESPONSE - This has not been done with other BMWG work items.  If it is
desired by the BMWG then it can be done.

13.  The device configuration,including Operating System scheduling and
timer modifications,
    features activated etc. would change the test results.

RESPONSE - Please review the Methodology and reply with any comments
specific to this concern or others.

14.      The benchmarks talk about offered load but do not clarify
    if the offered load is max load for the device or a
    smaller load. Seems like the relationship between offered
    load and max load would impact the results.

RESPONSE - Offered load is for an interface.

15.  the memo does not clarify is the type of traffic being offered
    It may be the case that the type of traffic may impact the
    convergence time due to the fact that additional processing
    resources are required on the device.

RESPONSE - Please review the Methodology and reply with any comments
specific to this concern or others.

16. traffic forwarding may be impacted by some other characteristic in
addition to route convergence.  The benchmarks would not distinguish those
cases clearly.
    However, at this point in time, I cannot think of any
    alternatives which would correct that. It may be sufficient
    to note that in the paper.

RESPONSE - Please review the Methodology and reply with any comments
specific to this concern or others.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Tuesday, June 08, 2004 8:56 PM
To: [email-masked]
Subject: [bmwg] Authors Action Items to Comments for
draft-ietf-bmwg-igp-datapla ne-conv-***-02

BMWG-ers,

Thanks very much to Brooks Hickman and  Robert Holley for their thorough
reviews of the Terminology document and to Tanju Cataltepe and Al Morton for
their thorough reviews of the Terminology and Methodology documents.  For
the sake of all BMWG participants I have consolidated and paraphrased their
comments for review.  Please let me know if you have additional or follow-up
comments.  The changes agreed to below will be submitted in 03 revisions
prior to the deadline for San Diego.  In addition, Bob Holley made further
comments that I will address in a separate email to follow shortly.

Scott
#################

TERMINOLOGY
General Comment - Al
Run IGP Draft Nits Tool. - Action Item ACCEPTED  

General Comment - Robert
Make capitalization of terms consistent. - Action Item ACCEPTED  

Section 1 Al and Robert (same comment)
Reformat Figure 1 in Introduction so that it displays AND prints correctly.
- Action Item ACCEPTED  

Section 3.1 
Tanju
Change definition of Convergence Event to be in terms of the externally
observable data plane. - Action Item ACCEPTED

Robert
Add 'Configuration Change to Convergence Event's list of example causes and
clarify 'better next-hop' as learned through receipt of a protocol update.
- Action Item ACCEPTED  

Section 3.2 Robert
Change Network Convergence so that it is not "approximated to the sum of
Route Convergence for all routers" since this approximation is too large and
Network Convergence is not yet defined. - Action Item ACCEPTED.  Is it
acceptable to swap the order of appearance for Route Convergence and Network
Convergence?  Instead of 'approximated to' is it acceptable to state
'bounded by'?

Section 3.5 Brooks
Question - Does the definition of Convergence Packet Loss include packets
that were lost and packets that were delayed due to buffering? -YES

Section 3.6 Tanju and Brooks (same comment)
Change definition of Convergence Event Instant to relate better to
Convergence Event as a data plane event. - Action Item ACCEPTED

Section 3.7 Brooks
Change discussion of Convergence Recovery Instant to explain external
observation of Full Convergence. - Action Item ACCEPTED

Section 3.14 Al
Change definition of Packet Sampling Interval to be an 'interval' while
providing comment about its use to determine 'rate'.  - Action Item ACCEPTED

Section 3.18 Brooks
Change definition of Preferred Egress Interface to be in terms of data plane
- Action Item ACCEPTED

Section 3.19 Brooks
Change definition of Next-Best Egress Interface to be in terms of data plane
- Action Item ACCEPTED

METHODOLOGY
General Comment - Al
Run IGP Draft Nits Tool - Action Item ACCEPTED  

Section 3.1 Al
Spell out ECMP - Action Item ACCEPTED

Section 3.2.3 Tanju
Expand discussion of IGP Route Scaling to include multiple areas and levels.
- Action Item ACCEPTED.  One question, Should the specific tests be
performed for a single area/level?  

Section 3.2.6 Tanju
Add not to the Offered Load that traffic must be sent to destinations
matching all IGP routes. - Action Item ACCEPTED.  

Section 3.3 Al
Provide Reporting Format for normalization when Recommended Values cannot be
used. - Action Item ACCEPTED. Excellent suggestion, but a this time it is
unknown to the authors how 'cleanly' this can be accomplished.  We will make
a concerted attempt at it.

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg