Subject: [bmwg] I-D Action: draft-ietf-bmwg-ipv6-tran-tech-benchmarking-01.txt

From: [email-masked]
Date: Wed, 16 Mar 2016 19:18:06 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology of the IETF.

        Title           : Benchmarking Methodology for IPv6 Transition Technologies
        Authors         : Marius Georgescu
                          Gabor Lencse
	Filename        : draft-ietf-bmwg-ipv6-tran-tech-benchmarking-01.txt
	Pages           : 25
	Date            : 2016-03-16

Abstract:
   There are benchmarking methodologies addressing the performance of
   network interconnect devices that are IPv4- or IPv6-capable, but the
   IPv6 transition technologies are outside of their scope. This
   document provides complementary guidelines for evaluating the
   performance of IPv6 transition technologies.  More specifically,
   this document targets IPv6 transition technologies that employ
   encapsulation or translation mechanisms, as dual-stack nodes can be
   very well tested using the recommendations of RFC2544 and RFC5180.
   The methodology also includes a tentative metric for benchmarking
   load scalability.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarking/

There's also a htmlized version available at:
https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-01

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-ipv6-tran-tech-benchmarking-01

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: Marius Georgescu <[email-masked]>
Date: Thu, 17 Mar 2016 12:08:24 +0900

Hello BMWG,

Thank you for your feedback and support so far. 
Please find below a changelog of version 01:

Following a suggestion from Scott Bradner
+ Terminology section for the terms and procedures defined in other RFCs
Following the discussion on the mailing list ( <http://www.ietf.org/mail-archive/web/bmwg/current/msg03371.html>http://www.ietf.org/mail-archive/web/bmwg/current/msg03371.html <http://www.ietf.org/mail-archive/web/bmwg/current/msg03371.html>):
+ New procedure for latency to report Typical Latency and Worst Latency
Following the ML discussion ( http://www.ietf.org/mail-archive/web/bmwg/current/msg03342.html <http://www.ietf.org/mail-archive/web/bmwg/current/msg03342.html> )  and empirical data 
+ Changed the average + moe to median + 1st-99th percentiles for summarizing & variation

We uploaded (here https://www.dropbox.com/sh/i1yh7nedw81xy92/AAAcxwFHK5t6E4oWHm1wGTK4a?dl=0 <https://www.dropbox.com/sh/i1yh7nedw81xy92/AAAcxwFHK5t6E4oWHm1wGTK4a?dl=0> ) some dns64 data and latency data as support for using median for summarizing, 1st-99th Percentiles for variation)
The DNS64 data was gathered using dns64perf++ (https://github.com/bakaid/dns64perfpp <https://github.com/bakaid/dns64perfpp>) for totd (https://github.com/fwdillema/totd <https://github.com/fwdillema/totd>) as DNS64 (https://tools.ietf.org/html/rfc6147 <https://tools.ietf.org/html/rfc6147>) forwarder and Bind9 (https://www.isc.org/downloads/bind/ <https://www.isc.org/downloads/bind/>) single thread as authoritative DNS server. 
The Latency data was gathered using D-ITG (http://traffic.comics.unina.it/software/ITG/ <http://traffic.comics.unina.it/software/ITG/>) for asamap (http://enog.jp/~masakazu/vyatta/map/ <http://enog.jp/~masakazu/vyatta/map/>) implementing MAP-T (https://tools.ietf.org/html/rfc7599 <https://tools.ietf.org/html/rfc7599>).
The graphs plot as alternatives for the summarizing function the median and the mean.
As for the variation, we plotted: the Margin of Error from the Average (99.9 confidence interval), Median Absolute Deviation, 1st-99th Percentiles. 

Following a suggestion from Fred Baker
+Simultaneous and Incremental benchmarks for network performance degradation

-Conclusion section
+ Added a table with the 4 defined generic categories and associated technologies proposed in the IETF

DNS Resolution Performance
 + Tester may be a single device or two physical devices
 + Test duration should be at least 60 seconds and timeout should be no more than 1 second, otherwise "gaming" is possible
 + Requirements for the Tester (including some explanations and a reference to dns64perf++)

*Still to cover
Suggestion from Fred Baker to have NATXX within the scope 

Best regards,
The authors

> On Mar 17, 2016, at 11:18, [email-masked] wrote:
> 
> 
> _______________________________________________
> bmwg mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/bmwg