Subject: [bmwg] draft-ietf-bmwg-firewall-06.txt Delta

From: "Hickman, Brooks" <[email-masked]>
Date: Wed, 25 Sep 2002 08:32:10 -0700

Below is a list of the deltas between revision 5 and revision 6 of the
draft-ietf-bmwg-firewall. Please review and provide any feedback. Thank you.

1) Added Section 4.10

   4.10 TCP Stack Considerations

   Some test instruments allow configuration of one or more TCP stack
   parameters, thereby influencing the traffic flows which will be
   offered and impacting performance measurements. While this document
   does not attempt to specify which TCP parameters should be
   configurable, any such TCP parameter(s) MUST be noted in the test
   report. In addition, when comparing multiple DUT/SUTs, the same TCP
   parameters MUST be used.

2) Added following to test parameters to Sections 5.4.2, 5.6.2.1, 5.8.2.1

   Close Method - Defines method for closing TCP connections. The test
   MUST be performed with either a three-way or four-way handshake. In
   a four-way handshake, each side sends separate FIN and ACK messages.
   In a three-way handshake, one side sends a combined FIN/ACK message
   upon receipt of a FIN.

   Close Direction - Defines whether closing of connections are to be
   initiated from the client or from the server.

3) Edited section 5.1.3

   WAS:

   When comparing DUT/SUTs with different MTUs, it is RECOMMENDED to
   limit the maximum IP size tested to the maximum MTU supported by all
   of the DUT/SUTs.

   IS:

   It is RECOMMENDED to perform the throughput measurements with
   different packet sizes. When testing with different packet sizes the
   DUT/SUT configuration MUST remain the same.

4) Edited section 5.2.3 and 5.3.3

   WAS:

   Between each iteration, it is RECOMMENDED that the tester issue a
   TCP RST referencing <all> connections attempted for the previous
   iteration, regardless of whether or not the connection attempt was
   successful.

   IS:

   Between each iteration, it is RECOMMENDED that the tester issue a
   TCP RST referencing each connection attempted for the previous
   iteration, regardless of whether or not the connection attempt was
   successful.

5) Edited Section 5.2.5.2

   Missing connection establishment time measurements

   WAS:

   The test report MUST note the connection attempt rate, age time and
   maximum concurrent connections measured. 

   IS:

   The test report MUST note the connection attempt rate, age time,
   minimum TCP connection establishment time, maximum TCP connection
   establishment time, average connection establishment time, aggregate
   connection establishment time and maximum concurrent connections
   measured.

6) Edited Section 5.3.5.2

   WAS:

   The test report MUST note the number of connections, age time and
   highest connection rate measured.

   IS:

   The test report MUST note the number of connections, age time, 
   minimum TCP connection establishment time, maximum TCP connection
   establishment time, average connection establishment time, aggregate
   connection establishment time and highest connection rate measured.

7) Edited section 5.4.3

   WAS:

   In the case of proxy based DUT/SUTs, the DUT/SUT will itself receive
   the final ACK in the three-way handshake when a connection is being
   torn down. For validation purposes, the virtual client(s) MAY
   verify that the DUT/SUT received the final ACK in the connection tear
   down exchange for all connections by transmitting a TCP datagram
   referencing the previously town down connection. A TCP RST should be
   received in response to the TCP datagram. 

   IS:

   In the case of proxy based DUT/SUTs, the DUT/SUT will itself receive
   the ACK in response to issuing a FIN packet to close its side of
   the TCP connection. For validation purposes, the virtual client or
   server, whichever is applicable, MAY verify that the DUT/SUT
   received the final ACK by re-transmitting the final ACK. A TCP RST
   should be received to the retransmitted ACK. 

8) Edited Section 5.4.5

   WAS:

   The test report MUST note the number of connections, age time and
   highest connection tear down rate measured.

   IS:

   The test report MUST note the number of connections, age time, close
   method, close direction, minimum TCP connection tear down time,
   maximum TCP connection tear down time, average TCP connection tear
   down time and the aggregate TCP connection tear down time and
   highest connection tear down rate measured.

9) Edited section 5.6.4.2

   Clarified transport layer measurements

   WAS:

   5.6.4.2 Measurements at or below the Transport-Layer

   The tester SHOULD make goodput[1] measurements for connection-
   oriented protocols at or  below the transport layer. Goodput
   measurements MUST only reference the protocols payload, excluding
   any of the protocols header. In addition, the tester MUST exclude
   any bits associated with the connection establishment, connection
   tear down, security associations or connection maintenance.

   Since connection-oriented protocols require that data be
   acknowledged, the offered load[6] will vary over the duration of the
   test. When performing forwarding rate measurements, the tester
   should measure the average forwarding rate over the duration of the
   test. 

   IS:

   5.6.4.2 Measurements at or below the Transport-Layer

   The following measurements SHOULD be performed for each connection-
   oriented protocol:

   Goodput[1] - Goodput as defined in section 3.17 of RFC2647.
   Measurements MUST only reference the protocol payload, excluding
   any of the protocol header. In addition, the tester MUST exclude
   any bits associated with the connection establishment, connection
   tear down, security associations or connection maintenance.
   Since connection-oriented protocols require that data be
   acknowledged, the offered load[6] will be varying. Therefore, the
   tester should measure the average forwarding rate over the
   duration of the test. Measurement should start on transmission of
   the first bit of the payload of the first datagram and end on
   transmission of the last bit of the payload of the last datagram.

   Number of bytes transferred - Total payload bytes transferred.

   Number of Timeouts - Total number of timeout events.

   Retransmitted bytes - Total number of retransmitted bytes. 

10) Edited section 5.6.5.2

   Edited based on deltas in measurements section 5.6.4.2

   WAS:

   5.6.5.2 Transport-Layer and below reporting

   The test report MUST note the aggregate number of connections. In
   addition, the report MUST identify the protocol for which the
   measurement was made. 

   The results SHOULD be in tabular form with a column for each
   iteration of the test. There should be columns for transmitted bits,
   retransmitted bits and the measured goodput.

   Failure analysis:

   The test report SHOULD indicate the number and percentage of
   connections that failed to complete.

   IS:

   5.6.5.2 Transport-Layer and below reporting

   The test report MUST note the number of connections, close method,
   close direction and the protocol for which the measurement was
   made. 

   The results SHOULD be reported in tabular form for each of the HTTP
   object sizes tested. There SHOULD be a row for the HTTP object size,
   resultant goodput, total timeouts, total retransmitted bytes and
   total bytes transferred. The table MAY be combined with the
   application layer reporting, provided the table clearly identify the
   protocol for which the measurement was made.

   Failure analysis:

   The test report SHOULD indicate the number and percentage of
   connection establishment failures as well as number and percentage
   of TCP tear down failures.

   It is RECOMMENDED that the report include a graph to plot the
   distribution of both connection establishment failures and
   connection tear down failures. The x coordinate SHOULD be the
   elapsed test time, the y coordinate SHOULD be the number of failures
   for a given sampling period. There SHOULD be two lines on the graph,
   one for connection failures and one for tear down failures. The
   graph MUST note the sampling period. 

11) Added section 5.7.5.2

   5.7.5.2 Transport-Layer

   The test report MUST note the close method, close direction, number
   of connections established and number of connections torn down.  

   The intermediate results of the search algorithm MAY be reported
   in a table format with a column for each iteration. There SHOULD be
   rows for the number of connections attempted, number and percentage
   of connections completed, number and percentage of connection tear
   downs completed. The table MAY be combined with the application
   layer reporting, provided the table identify this as transport layer
   measurement.

12) Added the following to section 5.10.3

   It is RECOMMENDED to perform the latency measurements with different
   packet sizes. When testing with different packet sizes the DUT/SUT
   configuration MUST remain the same.

13) Edited APPENDIX C

  WAS:

  When making connection tear down times measurements, such measurements
  will be made from the perspective of the client and will be performed
  in the same manner, independent of whether or not the DUT/SUT is
  proxy-based. The connection tear down will be considered the interval
  between the transmission of the first bit of the first TCP FIN packet
  transmitted by the tester requesting a connection tear down to receipt
  of the last bit of the corresponding ACK packet on the same tester
  interface.

  IS:

  When making connection tear down times measurements, such
  measurements will be made from the perspective of the entity - that
  is -- virtual client/server initiating the connection tear down
  request. In addition, the measurement will be performed in the same
  manner, independent of whether or not the DUT/SUT is proxy-based.
  The connection tear down will be considered the interval between the
  transmission of the first bit of the first TCP FIN packet transmitted
  by the virtual client or server, whichever is applicable, requesting
  a connection tear down to receipt of the last bit of the
  corresponding ACK packet on the same virtual client/server interface.

14) Removed HTTP Concurrent Transaction Capacity Test.

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg