Subject: [bmwg] Packet generator precision

From: Paul Emmerich <[email-masked]>
Date: Thu, 14 Jun 2018 22:11:12 +0200

Hi,

we've had a (very) short discussion on reliability and precision of software packet generators at the session at IETF 100.

I've wanted to revive this discussion because I might be at IETF 102 (travel grant application pending, fingers crossed),
if anyone here is still interested in this topic.

To get the discussion started, here's our research paper from last year where we looked at rate control of packet generators:
https://www.net.in.tum.de/fileadmin/bibtex/publications/papers/comparison_of_software_packet_generators.pdf

I want to draw your attention to Figure 1 on page 2. It shows a measurement of the median latency of packet forwarding with Open vSwitch
under increasing load. We repeated this test several times and modified the burst size of the packet generator while keeping the average
packet rate the same. The base line is true CBR traffic.

The result is an increase of the measured latency of up to 400% even though we only changed the spacing between packets on the wire.

Why is this important? Packet generators often generate small bursts in order to increase performance.
Usually the default is bursts of 16 to 256 packets and that can ruin latency tests in some cases.

The short discussion at IETF 100 was whether it might be helpful to have some kind of test for packet generators to measure their precision.

Paul

--
Chair of Network Architectures and Services
Department of Informatics
Technical University of Munich
Boltzmannstr. 3
85748 Garching bei München, Germany

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Thu, 14 Jun 2018 20:43:04 +0000

Hi Paul,

Thanks for your note and re-starting this discussion.

Post-IETF-100, BMWG had the opportunity to review the traffic
generator specifications in ETSI NFV draft specification, TST009.
https://docbox.etsi.org/ISG/NFV/Open/Drafts/TST009_NFVI_Benchmarks
I think what you have in mind goes beyond this specification in
several ways.

The Working group is in the luxurious position of considering 
new work, following the approval of our new charter. This gives
us flexibility to take-up obviously related items like generator
precision requirements, as long as there is interest in the 
working group.

I'll be re-reading your paper in the next few days, and 
good luck with your travel grant proposal!

regards,
Al
bmwg co-chair

> -----Original Message-----
> From: bmwg [mailto:[email-masked]] On Behalf Of Paul Emmerich
> Sent: Thursday, June 14, 2018 4:11 PM
> To: [email-masked]
> Subject: [bmwg] Packet generator precision
> 
> Hi,
> 
> we've had a (very) short discussion on reliability and precision of
> software packet generators at the session at IETF 100.
> 
> I've wanted to revive this discussion because I might be at IETF 102
> (travel grant application pending, fingers crossed),
> To get the discussion started, here's our research paper from last year
> where we looked at rate control of packet generators:
> I want to draw your attention to Figure 1 on page 2. It shows a
> measurement of the median latency of packet forwarding with Open vSwitch
> under increasing load. We repeated this test several times and modified
> the burst size of the packet generator while keeping the average
> The result is an increase of the measured latency of up to 400% even
> though we only changed the spacing between packets on the wire.
> 
> The short discussion at IETF 100 was whether it might be helpful to have
> some kind of test for packet generators to measure their precision.
> 
> 
> Paul
> 
> 
> --

---

From: Paul Emmerich <[email-masked]>
Date: Wed, 20 Jun 2018 17:43:30 +0200

> MORTON, ALFRED C (AL) <[email-masked]>:
> 
> Hi Paul,
> 

That looks interesting, I'll have a look at that.

> good luck with your travel grant proposal!

I'll be at IETF 102, ISOC invited me :)
I can give a short talk/summary at the meeting if you like.

Paul

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Wed, 20 Jun 2018 16:32:22 +0000

...
> I can give a short talk/summary at the meeting if you like.
[acm] 
Great, thanks.  Since all of our proposals end up in
Internet drafts at some point, think about how the doc
might be organized. That way if there is interest,
you could possibly divide the writing assignments among
an author-group, if formed.

Al