Subject: <draft-ietf-bmwg-mswitch-00.txt>

From: Kevin Dubray <[email-masked]>
Date: Fri, 07 Aug 1998 15:10:59 -0400

Network Working Group                                      R. Mandeville
Internet-Draft                             European Network Laboratories
Expiration Date: February 1999                                  J. Keene
                                                          Netcom Systems
                                                             August 1998

           Benchmarking Methodology for LAN Switching Devices
                    <draft-ietf-bmwg-mswitch-00.txt>

Status of this Memo

   This document is an Internet-Draft. Internet-Drafts are working
   documents of the Internet Engineering Task Force (IETF),its areas,
   and its working groups. Note that other groups may also distribute
   working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time. It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   To view the entire list of current Internet-Drafts, please check
   the "1id-abstracts.txt" listing contained in the Internet-Drafts
   Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net
   (Northern Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au
   (Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu
   (US West Coast).

   This memo provides information for the Internet community. This memo
   does not specify an Internet standard of any kind. Distribution of
   this memo is unlimited.

Abstract

   This document is intended to provide methodology for the benchmarking
   of local area network (LAN) switching devices.  It extends the
   methodology already defined for benchmarking network interconnecting
   devices in RFC 1944 to switching devices.

   This RFC primarily deals with devices which switch frames at the
   Medium Access Control (MAC) layer. It provides a methodology for
   benchmarking switching devices, forwarding performance, congestion
   control, latency, address handling and filtering. In addition to
   defining the tests, this document also describes specific formats for
   reporting the results of the tests.

1. Introduction

Mandeville, Keene                                               [Page 1]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

   This document defines a specific set of tests to measure and report
   the performance characteristics of network switching devices.

   A previous document, "Benchmarking Terminology for LAN Switching
   Devices" (RFC 2285), defined many of the terms that are used in this
   document.  The terminology document SHOULD be consulted before
   attempting to make use of this document.

2. Requirements

   The following RFCs SHOULD be consulted before attempting to make use
   of this document:

   * RFC 1242 "Benchmarking Terminology for Network Interconnect
     Devices"

   * RFC 1944 "Benchmarking Methodology for Network Interconnect
     Devices"

   * RFC 2285  "Benchmarking Terminology for LAN Switching Devices"

   For the sake of clarity and continuity, this RFC adopts the template
   for benchmarking tests set out in Section 26 of RFC 1944.

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in
   this document are to be interpreted as described in RFC 2119.

3. Test setup

   This document extends the general test setup described in section 6
   of RFC 1944 to the benchmarking of LAN switching devices. RFC 1944
   primarily describes non-meshed traffic where input and output
   interfaces  are grouped in mutually exclusive sending and
   receiving pairs. In fully meshed traffic, each interface of a
   DUT/SUT is set up to both receive and transmit frames to all
   the other interfaces under test.

   Prior to each test run, the DUT/SUT MUST learn the MAC addresses used
in
   the test and the address learning MUST be verified to avoid flooded
   frames being counted as correctly received frames. The forwarding
   rate, namely the rate at which address learning frames are offered
may
   have to be adjusted to be as low as 50 frames per second or even
less,
   to guarantee successful learning. The DUT/SUT address aging time
SHOULD be
   configured to be greater than the period between the learning phase
of
   the test and the test run; in non-meshed and partially meshed tests,
   the aging time SHOULD at a minimum be set to at least the length of
   the test period.  More than one trial may be needed for the
association

Mandeville, Keene                                               [Page 2]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

   of the address to the port to occur.

   If a DUT/SUT uses a hashing algorithm with address learning, the
   DUT/SUT may not learn the necessary addresses to perform the tests.
   The format of the MAC addresses MUST be adjustable so that the
address
   mapping may be re-arranged to make a DUT/SUT learn addresses
   without confusion.

   It is recommended that SNMP and Spanning Tree be disabled when bench-
   marking switching devices unless investigating overhead behavior. If
   such protocols cannot be turned off, it is recommended that the
   levels of offered load be reduced (less than 100%) to allow for
   the additional management frames.

4.  Frame formats and sizes

   For frame formats and sizes, refer to RFC 1944, sections 8 and 9 and
   Appendix C.

   There are three frame formats for layer 2 Ethernet switches:
   standard MAC Ethernet frames, standard MAC Ethernet frames
   with vendor-specific tags added to them, and IEEE 802.3ab
   frames tagged to accommodate 802.3p,q.  The two types of tagged
   frames may exceed the standard maximum length frame of 1518 bytes,
and
   may not be accepted by the interface controllers of some DUT/SUTs.
   It is recommended to check the compatibility of the DUT/SUT
   with tagged frames before testing.

   Devices switching tagged frames of over 1518 bytes will have a lower
   maximum forwarding rate than standard untagged frames.

5. Benchmarking Tests

   The following tests offer objectives, procedures, and reporting
   formats for benchmarking LAN switching devices.

     5.1  Fully meshed throughput, frame loss and forwarding rates
     5.2  Partially meshed overloading
     5.3  Head of line blocking
     5.4  Partially meshed multiple devices
     5.5  Multiple streams of unidirectional traffic
     5.6  Filter illegal frames
     5.7  Broadcast frame handling and latency
     5.8  Maximum forwarding rate and minimum interframe gap
     5.9  Address caching capacity
     5.10 Address learning rate

Mandeville, Keene                                               [Page 3]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

5.1  Fully meshed throughput, frame loss and forwarding rates

Objective:

   To determine the throughput, frame loss and forwarding rates of
DUT/SUTs
   offered fully meshed traffic as defined in RFC 2285.

Procedure:

   RFC 2285 points out that when offering bursty meshed traffic, the
   variables which MUST be defined are frame size, bust size,
   interframe gap, interburst gap, and load.  Each variable is
   configured with the following considerations.

      Interframe Gap (IFG)  -  The IFG between frames inside a burst
MUST
      be at the minimum specified by the standard (9.6 us for 10Mbps
      Ethernet and 0.96 us for 100Mbps Ethernet).

      Interburst Gap (IBG) - This is the interval between bursts of
      traffic.  Refer to Appendix A, Calculating Interburst Gap, for
      the formula used to compute IBG.

      Load / Port - The test SHOULD be run multiple times with a
      different load per port in each case. The 100% load translates to
      a transmit load of 50% for half duplex. This type of test SHOULD
      also be run at higher than 100% loads.

      In half duplex mode, exactly half of the target load SHOULD be
      sent to each of the ports under test. For example, with a 100%
      load of 64-byte frames, the target load for each port under test
      is 7440 frames received per second and 7440 frames transmitted per
      second (for 10Mbps Ethernet).

      Frame Size - Recommended frame sizes are 64, 128, 256, 512, 1024,
      1280 and 1518 bytes, per RFC 1944 section 9.

      Burst Size - The burst size defines the number of packets sent
      back-to-back at the minimum legal IFG (96 bit times) before
      pausing transmission to receive frames.  Burst sizes
      SHOULD vary between 1 and 930 frames.

      To avoid truncating bursts, a burst size SHOULD be an even
      multiple into 7440. 7440 is the maximum frames per second for half
      duplex mode; 14880 is the maximum frames per second for full
      duplex mode.  Refer to Appendix A for a table of recommended burst
      values.

Mandeville, Keene                                               [Page 4]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

   Each port in the test sends frames to all other ports in a round-
   robin type fashion. The following table shows how each port in a test
   SHOULD transmit frames to all other ports in the test. In this
   example, there are six ports with 1 address per port:

   Source Port, then Destination Ports (in order of transmission)

   Port #1              2       3       4       5       6       2...
   Port #2              3       4       5       6       1       3...
   Port #3              4       5       6       1       2       4...
   Port #4              5       6       1       2       3       5...
   Port #5              6       1       2       3       4       6...
   Port #6              1       2       3       4       5       1...

   As shown in the table, there is an equal distribution of destination
   addresses for each transmit opportunity. This keeps the test balanced
   so that one destination port is not overloaded by the test algorithm
   and all ports are equally and fully loaded throughout the test. For
   tests using multiple addresses per port, the actual port destinations
   are the same as described above and the actual source/destination
address
   pairs are chosen randomly to exercise the DUT/SUT's ability to
perform
   address lookups.

   For every address, the testing device sends learning packets to allow
   the DUT/SUT to load its address tables properly.

   To measure the DUT/SUT's ability to switch traffic while performing
many
   different address lookups, the number of addresses per port SHOULD be
   increased in a series of tests.

Reporting format:

   In these tests, a port SHOULD transmit and receive the same amount of
   packets. Each port MUST count the packets received with a valid
address.
   Any packet received which does not have a valid address MUST not be
   counted as a received packet and can be counted as part of a flood
   count as described in 3.8.3 in RFC 2285.

   The results for these tests SHOULD be reported in the form of
numerical
   data or a graph with text to indicate the data type. The data types
for
   the throughput test and for the frame loss rate test are described in
   RFC 1944.

   Forwarding rate (FR) of the DUT/SUT SHOULD be reported as the number
of
   frames per second that the device is observed to successfully
transmit
   to the correct destination interface in response to a specified
offered
   load. The offered load MUST also be cited.

Mandeville, Keene                                               [Page 5]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

   Forwarding rate at maximum offered load (FRMOL ) SHOULD be reported
as
   the number of frames per second that a device can successfully
transmit
   to the correct destination interface in response to the maximum
offered
   load as defined in RFC 2285, section 3.6. The maximum offered load
MUST
   also be cited.

   Maximum forwarding rate (MFR) SHOULD be reported as the highest
forwarding
   rate of a DUT/SUT taken from an iterative set of forwarding rate
   measurements. The load applied to the device MUST also be cited.

5. 2  Partially meshed overloading

To be done.

5. 3  Head-of-line blocking

To be done.

5.4   Partially Meshed Multiple Devices

To be done.

5.5   Multiple streams of unidirectional traffic

To be done.

5.6   Filter illegal frames

To be done.

5.7   Broadcast frame handling and latency test

To be done.

5.8   Maximum forwarding rate and minimum interframe gap

To be done.

5.9   Address Caching Capacity

Objective:

   To determine the address caching capacity of a LAN switching device
   as defined in RFC 2285, section 3.8.1.

Procedure:

Mandeville, Keene                                               [Page 6]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

   This test SHOULD at a minimum be performed in a three-port
   configuration as described below.

   The first port (port 1) of the testing device is connected to the
DUT/SUT
   and is the port from which a testing device sends frames with varying
   source addresses and a fixed destination address corresponding to the
MAC
   address of the receiving port. By receiving frames with varying
source
   addresses, the DUT/SUT will learn these new addresses from the
sending port
   of the test device.

   A second port (port 2) of the testing device is connected to the
DUT/SUT
   and acts as the receiving port for the address learning frames. This
   port also sends "control" frames back to the addresses learned
   on the first port. The algorithm for this is explained below.

   --  A third port (port 3) on the testing device MUST be connected to
a
   port on the DUT/SUT and act as a monitoring port to listen for
   flooded frames.

   The algorithm for the test is as follows:

   BEGIN
      Set Initial Value of N to user specified number, where N is the
      number of addresses to be verified in each iteration
    WHILE NOT Finished DO
      PAUSE for the aging time specified
      Address learning:  Port 1 sends N frames with varying
         source addresses to Port 2 to attempt to fill the DUT/SUT
         address table for Port #1.
      Controlling:  Port 2 sends N frames with varying
         destination addresses corresponding to Port 1.
      IF (Port 3 received a frame during Control phase) OR
         (Port 1 did not receive the correct # of frames)
      THEN
         Address Table of DUT/SUT was full
         Set N to lower number (in binary search method)
      ELSE
         Address Table of DUT/SUT was NOT full
         Set N to higher number (in binary search method)
      IF High and Low Values of N Meet
      THEN
         Test is Finished, Value of N equals number of
           addresses supported by DUT/SUT
      ELSE
         Continue Test
     END WHILE
   DONE

Mandeville, Keene                                               [Page 7]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

   Using a binary search approach, the test targets the exact number
   of addresses supported per port with minimal test iterations. Due
   to the aging time of DUT/SUT address tables, each iteration may take
   some time during the waiting period for the addresses to clear. If
   possible, configure the DUT/SUT for a low value for the aging time.

   Once the high and low values of N meet, then the threshold of address
   handling has been found.

   For this test, the aging time of the DUT/SUT MUST be known. Use a
short
   aging time if possible to reduce the time needed to run the test.
   The aging time MUST be longer than the time necessary to produce
   packets at the specified rate. If a low frame rate is used for
   the test, then it may be possible that sending a large amount of
   frames may actually take longer than the aging time. Keep in mind
   that the test actually sends twice as many frames as the number
   of addresses being tested due to the learning phase and the
   controlling phase.

   Set the initial value of addresses per port to a number
   slightly higher than the number of addresses that the DUT/SUT can
handle.
   This step primes the binary search and can reduce the number of
   iterations required to determine the exact number supported.

   Set the forwarding rate to a number that is reasonable to be handled
   by the DUT/SUT and one that is high enough so that the test
iterations are
   not too long.

Reporting format:

   After the test is run, results for each iteration SHOULD be displayed
   in a table to include:

   --  the number of addresses used for each test iteration.

   --  the frame rate used for each test iteration.

   --  number of control frames that were transmitted by test port
   number 2. Control frames are the frames sent with varying destination
   addresses to confirm that the DUT/SUT has learned all of the
addresses
   for each test iteration.

   --  the number of frames received by test port 2 during the control
   portion of each test. If the number is non-zero, this is an
indication
   of the DUT/SUT flooding a frame in which the destination address is
not
   in the address table.

   --  the number of frames that test port 1 received during the control

Mandeville, Keene                                               [Page 8]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

   portion of the test. This number will include those frames which are
   Spanning tree frames. For a normal test iteration, this number SHOULD
   be equal to the number of frames transmitted by port 2 during the
   control phase.

   --  the number of frames that test port 1 received during the control
   phase not including Spanning Tree or other non test generated frames.
   In all cases, this number SHOULD be equal to the number of frames
   transmitted by port 2 during the control phase of the test.

   --  the number of frames test port 3 received during the control
phase
   of the test. If the value is not zero, then this indicates that for
   that test iteration, the DUT/SUT could not determine the proper
destination
   port for that many frames. In other words, the DUT/SUT flooded the
frame to
   all ports since its address table was full.

5.10   Address Learning Rate

Objective:

   Once the maximum number of addresses supported per port by the
DUT/SUT is
   known, the addressing learning rate (the rate at which the DUT/SUT
learns
   these addresses) can be determined.

Procedure:

   An algorithm similar to the one used to determine address caching
   capacity can be used to determine the address learning rate. This
   test iterates the rate at which address learning frames are offered
   by the test device connected to the DUT/SUT.  It is recommended to
set
   the number of addresses offered to the DUT/SUT in this test to the
   maximum caching capacity.  However, the address learning rate might
be
   determined for different numbers of addresses but in each test run,
   the number MUST remain constant.

   Initializing the forwarding rate primes the binary search algorithm
   and can help to shorten the overall test duration.

   A third port on the DUT/SUT MUST listen for flooded frames.

   In this test, the forwarding rate SHOULD be varied until a rate is
   found that is the threshold for the rate supported by the DUT/SUT. It
   may be useful to pick a value slightly higher than the advertised
   forwarding rate.

Reporting format:

Mandeville, Keene                                               [Page 9]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

   To be done.

Section 6. Security Considerations

   This document does not yet address Security Considerations.

Section 7. Authors' Address

   Robert Mandeville
   European Network Laboratories (ENL)
   2, rue Helene Boucher
   87286 Guyancourt Cedex
   France

   Phone: + 33 1 39 44 12 05
   EMail: [email-masked]

   Judy Keene
   Netcom Systems
   20550 Nordhoff St.
   Chatsworth, CA 91311
   USA

   Phone: +1 818 700 5100
   Email: [email-masked]

Appendix A:  Calculating the Interburst Gap

   IBG is defined in RFC 2285 as a function of maximum media rate (also
   known as line rate), the length of the frames in the bursts with the
   preamble and the interframe gap (IFG), the number of frames in the
bursts,
   and the intended load.

   Using the burst size, frame size and the load per port, the IBG can
be
   calculated:

     Burst size * ((preamble 64 + (frame size * 8 bits) + 96 IFG bit
times))
             * 1 (for full duplex) or .5 (for half duplex)
            (10 bit times equal 1 microsecond)

   Example:

      Burst size = 24

Mandeville, Keene                                              [Page 10]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

      Frame size = 64

        24 * (64 + 64*8 + 96) = 24 * (672) = 16,128 bit/times =
                                                 1612.8 us IBG

   In half duplex mode, exactly half of the target load SHOULD be sent
   to each of the ports under test. For example, with a 100% load of
   64-byte frames, the target load for each port under test is 7440
   frames received per second and 7440 frames transmitted per second
   (for 10Mbps Ethernet).

   Recommended characteristic values for burst size are:

2       15      60      240     930
3       16      62      248
4       20      80      310
5       24      93      372
6       30      120     465
8       31      124     496
10      40      155     620
12      48      186     744

Mandeville, Keene                                              [Page 11]

Network Working Group                                      R. Mandeville
Internet-Draft                             European Network Laboratories
Expiration Date: February 1999                                  J. Keene
                                                          Netcom Systems
                                                             August 1998

           Benchmarking Methodology for LAN Switching Devices
                    <draft-ietf-bmwg-mswitch-00.txt>

Status of this Memo

   This document is an Internet-Draft. Internet-Drafts are working
   documents of the Internet Engineering Task Force (IETF),its areas,
   and its working groups. Note that other groups may also distribute
   working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time. It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   To view the entire list of current Internet-Drafts, please check
   the "1id-abstracts.txt" listing contained in the Internet-Drafts
   Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net
   (Northern Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au
   (Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu
   (US West Coast).

   This memo provides information for the Internet community. This memo
   does not specify an Internet standard of any kind. Distribution of
   this memo is unlimited.

Abstract

   This document is intended to provide methodology for the benchmarking
   of local area network (LAN) switching devices. It extends the
   methodology already defined for benchmarking network interconnecting
   devices in RFC 1944 to switching devices.

   This RFC primarily deals with devices which switch frames at the
   Medium Access Control (MAC) layer. Itprovides a methodology for
   benchmarking switching devices, forwarding performance, congestion
   control, latency, address handling and filtering. In addition to
   defining the tests, this document also describes specific formats for
   reporting the results of the tests.

1. Introduction

Mandeville, Keene                                               [Page 1]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

   This document defines a specific set of tests to measure and report
   the performance characteristics of network switching devices.

   A previous document, "Benchmarking Terminology for LAN Switching
   Devices" (RFC 2285), defined many of the terms that are used in this
   document.  The terminology document SHOULD be consulted before
   attempting to make use of this document.

2. Requirements

   The following RFCs SHOULD be consulted before attempting to make use
   of this document:

   * RFC 1242 "Benchmarking Terminology for Network Interconnect
     Devices"

   * RFC 1944 "Benchmarking Methodology for Network Interconnect
     Devices"

   * RFC 2285  "Benchmarking Terminology for LAN Switching Devices"

   For the sake of clarity and continuity, this RFC adopts the template
   for benchmarking tests set out in Section 26 of RFC 1944.

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in
   this document are to be interpreted as described in RFC 2119.

3. Test setup

   This document extends the general test setup described in section 6
   of RFC 1944 to the benchmarking of LAN switching devices. RFC 1944
   primarily describes non-meshed traffic where input and output
   interfaces  are grouped in mutually exclusive sending and
   receiving pairs. In fully meshed traffic, each interface of a
   DUT/SUT is set up to both receive and transmit frames to all
   the other interfaces under test.

   Prior to each test run, the DUT/SUT MUST learn the MAC addresses used
in
   the test and the address learning MUST be verified to avoid flooded
   frames being counted as correctly received frames. The forwarding
   rate, namely the rate at which address learning frames are offered
may
   have to be adjusted to be as low as 50 frames per second or even
less,
   to guarantee successful learning. The DUT/SUT address aging time
SHOULD be
   configured to be greater than the period between the learning phase
of
   the test and the test run; in non-meshed and partially meshed tests,
   the aging time SHOULD at a minimum be set to at least the length of
   the test period.  More than one trial may be needed for the
association

Mandeville, Keene                                               [Page 2]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

   of the address to the port to occur.

   If a DUT/SUT uses a hashing algorithm with address learning, the
   DUT/SUT may not learn the necessary addresses to perform the tests.
   The format of the MAC addresses MUST be adjustable so that the
address
   mapping may be re-arranged to make a DUT/SUT learn addresses
   without confusion.

   It is recommended that SNMP and Spanning Tree be disabled when bench-
   marking switching devices unless investigating overhead behavior. If
   such protocols cannot be turned off, it is recommended that the
   levels of offered load be reduced (less than 100%) to allow for
   the additional management frames.

4.  Frame formats and sizes

   For frame formats and sizes, refer to RFC 1944, sections 8 and 9 and
   Appendix C.

   There are three frame formats for layer 2 Ethernet switches:
   standard MAC Ethernet frames, standard MAC Ethernet frames
   with vendor-specific tags added to them, and IEEE 802.3ab
   frames tagged to accommodate 802.3p,q.  The two types of tagged
   frames may exceed the standard maximum length frame of 1518 bytes,
and
   may not be accepted by the interface controllers of some DUT/SUTs.
   It is recommended to check the compatibility of the DUT/SUT
   with tagged frames before testing.

   Devices switching tagged frames of over 1518 bytes will have a lower
   maximum forwarding rate than standard untagged frames.

5. Benchmarking Tests

   The following tests offer objectives, procedures, and reporting
   formats for benchmarking LAN switching devices.

     5.1  Fully meshed throughput, frame loss and forwarding rates
     5.2  Partially meshed overloading
     5.3  Head of line blocking
     5.4  Partially meshed multiple devices
     5.5  Multiple streams of unidirectional traffic
     5.6  Filter illegal frames
     5.7  Broadcast frame handling and latency
     5.8  Maximum forwarding rate and minimum interframe gap
     5.9  Address caching capacity
     5.10 Address learning rate

Mandeville, Keene                                               [Page 3]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

5.1  Fully meshed throughput, frame loss and forwarding rates

Objective:

   To determine the throughput, frame loss and forwarding rates of
DUT/SUTs
   offered fully meshed traffic as defined in RFC 2285.

Procedure:

   RFC 2285 points out that when offering bursty meshed traffic, the
   variables which MUST be defined are frame size, bust size,
   interframe gap, interburst gap, and load.  Each variable is
   configured with the following considerations.

      Interframe Gap (IFG)  -  The IFG between frames inside a burst
MUST
      be at the minimum specified by the standard (9.6 us for 10Mbps
      Ethernet and 0.96 us for 100Mbps Ethernet).

      Interburst Gap (IBG) - This is the interval between bursts of
      traffic.  Refer to Appendix A, Calculating Interburst Gap, for
      the formula used to compute IBG.

      Load / Port - The test SHOULD be run multiple times with a
      different load per port in each case. The 100% load translates to
      a transmit load of 50% for half duplex. This type of test SHOULD
      also be run at higher than 100% loads.

      In half duplex mode, exactly half of the target load SHOULD be
      sent to each of the ports under test. For example, with a 100%
      load of 64-byte frames, the target load for each port under test
      is 7440 frames received per second and 7440 frames transmitted per
      second (for 10Mbps Ethernet).

      Frame Size - Recommended frame sizes are 64, 128, 256, 512, 1024,
      1280 and 1518 bytes, per RFC 1944 section 9.

      Burst Size - The burst size defines the number of packets sent
      back-to-back at the minimum legal IFG (96 bit times) before
      pausing transmission to receive frames.  Burst sizes
      SHOULD vary between 1 and 930 frames.

      To avoid truncating bursts, a burst size SHOULD be an even
      multiple into 7440. 7440 is the maximum frames per second for half
      duplex mode; 14880 is the maximum frames per second for full
      duplex mode.  Refer to Appendix A for a table of recommended burst
      values.

Mandeville, Keene                                               [Page 4]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

   Each port in the test sends frames to all other ports in a round-
   robin type fashion. The following table shows how each port in a test
   SHOULD transmit frames to all other ports in the test. In this
   example, there are six ports with 1 address per port:

   Source Port, then Destination Ports (in order of transmission)

   Port #1              2       3       4       5       6       2...
   Port #2              3       4       5       6       1       3...
   Port #3              4       5       6       1       2       4...
   Port #4              5       6       1       2       3       5...
   Port #5              6       1       2       3       4       6...
   Port #6              1       2       3       4       5       1...

   As shown in the table, there is an equal distribution of destination
   addresses for each transmit opportunity. This keeps the test balanced
   so that one destination port is not overloaded by the test algorithm
   and all ports are equally and fully loaded throughout the test. For
   tests using multiple addresses per port, the actual port destinations
   are the same as described above and the actual source/destination
address
   pairs are chosen randomly to exercise the DUT/SUT's ability to
perform
   address lookups.

   For every address, the testing device sends learning packets to allow
   the DUT/SUT to load its address tables properly.

   To measure the DUT/SUT's ability to switch traffic while performing
many
   different address lookups, the number of addresses per port SHOULD be
   increased in a series of tests.

Reporting format:

   In these tests, a port SHOULD transmit and receive the same amount of
   packets. Each port MUST count the packets received with a valid
address.
   Any packet received which does not have a valid address MUST not be
   counted as a received packet and can be counted as part of a flood
   count as described in 3.8.3 in RFC 2285.

   The results for these tests SHOULD be reported in the form of
numerical
   data or a graph with text to indicate the data type. The data types
for
   the throughput test and for the frame loss rate test are described in
   RFC 1944.

   Forwarding rate (FR) of the DUT/SUT SHOULD be reported as the number
of
   frames per second that the device is observed to successfully
transmit
   to the correct destination interface in response to a specified
offered
   load. The offered load MUST also be cited.

Mandeville, Keene                                               [Page 5]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

   Forwarding rate at maximum offered load (FRMOL ) SHOULD be reported
as
   the number of frames per second that a device can successfully
transmit
   to the correct destination interface in response to the maximum
offered
   load as defined in RFC 2285, section 3.6. The maximum offered load
MUST
   also be cited.

   Maximum forwarding rate (MFR) SHOULD be reported as the highest
forwarding
   rate of a DUT/SUT taken from an iterative set of forwarding rate
   measurements. The load applied to the device MUST also be cited.

5. 2  Partially meshed overloading

To be done.

5. 3  Head-of-line blocking

To be done.

5.4   Partially Meshed Multiple Devices

To be done.

5.5   Multiple streams of unidirectional traffic

To be done.

5.6   Filter illegal frames

To be done.

5.7   Broadcast frame handling and latency test

To be done.

5.8   Maximum forwarding rate and minimum interframe gap

To be done.

5.9   Address Caching Capacity

Objective:

   To determine the address caching capacity of a LAN switching device
   as defined in RFC 2285, section 3.8.1.

Procedure:

Mandeville, Keene                                               [Page 6]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

   This test SHOULD at a minimum be performed in a three-port
   configuration as described below.

   The first port (port 1) of the testing device is connected to the
DUT/SUT
   and is the port from which a testing device sends frames with varying
   source addresses and a fixed destination address corresponding to the
MAC
   address of the receiving port. By receiving frames with varying
source
   addresses, the DUT/SUT will learn these new addresses from the
sending port
   of the test device.

   A second port (port 2) of the testing device is connected to the
DUT/SUT
   and acts as the receiving port for the address learning frames. This
   port also sends "control" frames back to the addresses learned
   on the first port. The algorithm for this is explained below.

   --  A third port (port 3) on the testing device MUST be connected to
a
   port on the DUT/SUT and act as a monitoring port to listen for
   flooded frames.

   The algorithm for the test is as follows:

   BEGIN
      Set Initial Value of N to user specified number, where N is the
      number of addresses to be verified in each iteration
    WHILE NOT Finished DO
      PAUSE for the aging time specified
      Address learning:  Port 1 sends N frames with varying
         source addresses to Port 2 to attempt to fill the DUT/SUT
         address table for Port #1.
      Controlling:  Port 2 sends N frames with varying
         destination addresses corresponding to Port 1.
      IF (Port 3 received a frame during Control phase) OR
         (Port 1 did not receive the correct # of frames)
      THEN
         Address Table of DUT/SUT was full
         Set N to lower number (in binary search method)
      ELSE
         Address Table of DUT/SUT was NOT full
         Set N to higher number (in binary search method)
      IF High and Low Values of N Meet
      THEN
         Test is Finished, Value of N equals number of
           addresses supported by DUT/SUT
      ELSE
         Continue Test
     END WHILE
   DONE

Mandeville, Keene                                               [Page 7]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

   Using a binary search approach, the test targets the exact number
   of addresses supported per port with minimal test iterations. Due
   to the aging time of DUT/SUT address tables, each iteration may take
   some time during the waiting period for the addresses to clear. If
   possible, configure the DUT/SUT for a low value for the aging time.

   Once the high and low values of N meet, then the threshold of address
   handling has been found.

   For this test, the aging time of the DUT/SUT MUST be known. Use a
short
   aging time if possible to reduce the time needed to run the test.
   The aging time MUST be longer than the time necessary to produce
   packets at the specified rate. If a low frame rate is used for
   the test, then it may be possible that sending a large amount of
   frames may actually take longer than the aging time. Keep in mind
   that the test actually sends twice as many frames as the number
   of addresses being tested due to the learning phase and the
   controlling phase.

   Set the initial value of addresses per port to a number
   slightly higher than the number of addresses that the DUT/SUT can
handle.
   This step primes the binary search and can reduce the number of
   iterations required to determine the exact number supported.

   Set the forwarding rate to a number that is reasonable to be handled
   by the DUT/SUT and one that is high enough so that the test
iterations are
   not too long.

Reporting format:

   After the test is run, results for each iteration SHOULD be displayed
   in a table to include:

   --  the number of addresses used for each test iteration.

   --  the frame rate used for each test iteration.

   --  number of control frames that were transmitted by test port
   number 2. Control frames are the frames sent with varying destination
   addresses to confirm that the DUT/SUT has learned all of the
addresses
   for each test iteration.

   --  the number of frames received by test port 2 during the control
   portion of each test. If the number is non-zero, this is an
indication
   of the DUT/SUT flooding a frame in which the destination address is
not
   in the address table.

   --  the number of frames that test port 1 received during the control

Mandeville, Keene                                               [Page 8]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

   portion of the test. This number will include those frames which are
   Spanning tree frames. For a normal test iteration, this number SHOULD
   be equal to the number of frames transmitted by port 2 during the
   control phase.

   --  the number of frames that test port 1 received during the control
   phase not including Spanning Tree or other non test generated frames.
   In all cases, this number SHOULD be equal to the number of frames
   transmitted by port 2 during the control phase of the test.

   --  the number of frames test port 3 received during the control
phase
   of the test. If the value is not zero, then this indicates that for
   that test iteration, the DUT/SUT could not determine the proper
destination
   port for that many frames. In other words, the DUT/SUT flooded the
frame to
   all ports since its address table was full.

5.10   Address Learning Rate

Objective:

   Once the maximum number of addresses supported per port by the
DUT/SUT is
   known, the addressing learning rate (the rate at which the DUT/SUT
learns
   these addresses) can be determined.

Procedure:

   An algorithm similar to the one used to determine address caching
   capacity can be used to determine the address learning rate. This
   test iterates the rate at which address learning frames are offered
   by the test device connected to the DUT/SUT.  It is recommended to
set
   the number of addresses offered to the DUT/SUT in this test to the
   maximum caching capacity.  However, the address learning rate might
be
   determined for different numbers of addresses but in each test run,
   the number MUST remain constant.

   Initializing the forwarding rate primes the binary search algorithm
   and can help to shorten the overall test duration.

   A third port on the DUT/SUT MUST listen for flooded frames.

   In this test, the forwarding rate SHOULD be varied until a rate is
   found that is the threshold for the rate supported by the DUT/SUT. It
   may be useful to pick a value slightly higher than the advertised
   forwarding rate.

Reporting format:

Mandeville, Keene                                               [Page 9]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

   To be done.

Section 6. Security Considerations

   This document does not yet address Security Considerations.

Section 7. Authors' Address

   Robert Mandeville
   European Network Laboratories (ENL)
   2, rue Helene Boucher
   87286 Guyancourt Cedex
   France

   Phone: + 33 1 39 44 12 05
   EMail: [email-masked]

   Judy Keene
   Netcom Systems
   20550 Nordhoff St.
   Chatsworth, CA 91311
   USA

   Phone: +1 818 700 5100
   Email: [email-masked]

Appendix A:  Calculating the Interburst Gap

   IBG is defined in RFC 2285 as a function of maximum media rate (also
   known as line rate), the length of the frames in the bursts with the
   preamble and the interframe gap (IFG), the number of frames in the
bursts,
   and the intended load.

   Using the burst size, frame size and the load per port, the IBG can
be
   calculated:

     Burst size * ((preamble 64 + (frame size * 8 bits) + 96 IFG bit
times))
             * 1 (for full duplex) or .5 (for half duplex)
            (10 bit times equal 1 microsecond)

   Example:

      Burst size = 24

Mandeville, Keene                                              [Page 10]

INTERNET-DRAFT    LAN Switch Benchmarking Methodology        August 1998

      Frame size = 64

        24 * (64 + 64*8 + 96) = 24 * (672) = 16,128 bit/times =
                                                 1612.8 us IBG

   In half duplex mode, exactly half of the target load SHOULD be sent
   to each of the ports under test. For example, with a 100% load of
   64-byte frames, the target load for each port under test is 7440
   frames received per second and 7440 frames transmitted per second
   (for 10Mbps Ethernet).

   Recommended characteristic values for burst size are:

2       15      60      240     930
3       16      62      248
4       20      80      310
5       24      93      372
6       30      120     465
8       31      124     496
10      40      155     620
12      48      186     744

Mandeville, Keene                                              [Page 11]