Subject: RE: [bmwg] Terminology Question: Throughput

From: "Perser, Jerry" <[email-masked]>
Date: Thu, 13 Nov 2003 12:24:39 -0800

Okay,

Let me restate the problem assuming the loss detection happens at both the
ingress and egress:

Throughput is a measured RATE at the ingress of the DUT/SUT [RFC1242]
Forwarding Rate is a measured RATE at the egress of the DUT/SUT [RFC1242]

As I understand Timmons question on ipsec I-D:

9.3.1 IPsec Tunnel Throughput
   Definition:
      The forwarding rate through an IKE/IPsec tunnel at which none of
      the offered frames are dropped by the device under test.

Does the measurement Units follow RFC1242 or RFC2285?

Jerry.

-----Original Message-----
From: Jim McQuaid [mailto:[email-masked]]
Sent: Thursday, November 13, 2003 11:53 AM
To: 'John Dawson'; [email-masked]'
Subject: RE: [bmwg] Terminology Question: Throughput

Yes, correct.  And you state it better than I did.  You have to *measure* on
both ingress and egress, but the metric is the ingress rate at which the
throughput condition is satisfied.

There was a discussion years back about "goodput" (and other suggested
nomenclature) which was kind of throughput with some amount of dropped
frames allowed or ignored or deducted in some fashion.  But that discussion
never yielded any consensus about what is acceptable or useful in practice
when it comes to dropping frames.
-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of John
Dawson
Sent: Thursday, November 13, 2003 2:46 PM
To: [email-masked]'
Subject: RE: [bmwg] Terminology Question: Throughput

I agree with Jerry on this one.  The throughput measurement as not as clear
as you think (Jim). 
Yes, you do look at the number of packets/frames sent (ingress) and compare
it to the number received (egress), and you try to find a maximum rate that
can be offered on the ingress without loss.
However, when it comes time to report the final "rate", you must report the
offered load -- which is purely an ingress calculation.  The reported
Throughput rate has nothing to due with the received (egress) rate, so long
as all packets are received.
John D. 
-----Original Message----- 
From: Jim McQuaid [mailto:[email-masked]] 
Sent: Thursday, November 13, 2003 11:26 AM 
To: 'Perser, Jerry'; [email-masked]' 
Subject: RE: [bmwg] Terminology Question: Throughput 

No.  You cannot measure throughput without measuring ingress and egress. 
The definition of throughput perhaps allows some ambiguity.  However, how 
would one detect "dropped" if only looking at ingress?  Likewise, the 
definition of frame loss rate clearly talks about forwarded (egress) frames;

the 'percentage' can only be calculated if you have both ingress and egress.

Finally, the definition of the throughput procedure is extremely clear about

HOW to measure throughput: by counting ingress and egress and comparing it. 
Test equipment implementing this test did exactly that.  It is necessary to 
achieve a sufficently long test duration to overcome buffering effects, 
hence the reference in 'frame loss rate' to "steady state." 
You said that your issue is that 'throughput' cannot be both an ingress and 
egress metric.  In one sense, that's true.  It's a metric about throughput, 
namely the relationship of inputs to outputs and it requires two 
simultaneous measurements to be computed. 
cheers, 
Jim McQuaid 
NetIQ Raleigh 
------------------------------------------ 
1242 definition of throughput: 
"The maximum rate at which none of the offered frames are dropped by the 
device." 
1242 definition of frame loss rate: 
"Percentage of frames that should have been forwarded by a network device 
under steady state (constant)load that were not forwarded due to lack of 
resources." 
1944/2544 definition of throughput benchmark procedure: 
 'Procedure:  Send a specific number of frames at a specific rate through 
the DUT and then count the frames that are transmitted by the DUT. ' 

-----Original Message----- 
From: Perser, Jerry [mailto:[email-masked]] 
Sent: Thursday, November 13, 2003 12:38 PM 
To: Jim McQuaid; [email-masked]' 
Subject: RE: [bmwg] Terminology Question: Throughput 

Jim, 
The definition leads one to believe that it is an ingress method.  "The 
maximum rate at which none of the offered frames are dropped by the device" 
[RFC1242] mentions offered frames, not forwarded frames.  Also the 
measurement units are in N-octet input frames per second.  IMHO, ingress and

input are the same. 
My issue is that throughput can not both be an ingress metric AND an egress 
metric.  In any DUT/SUT that buffers (store and forward devices) the maximum

zero loss rate at the ingress will be higher than the egress.  Vendors will 
use this ambiguity to engage in "specsmanship". 
Jerry. 
> -----Original Message----- 
> From: Jim McQuaid [mailto:[email-masked]] 
> Sent: Thursday, November 13, 2003 5:52 AM 
> To: 'Perser, Jerry'; [email-masked]' 
> Subject: RE: [bmwg] Terminology Question: Throughput 
> 
> 
> I think the concept that throughput can be measured at the ingress of 
> anything is incorrect.  Throughput is the ability of a DUT to 
> forward 100% 
> of the offered load.  The definition more or less requires 
> measurement on 
> both the ingress and egress and a comparison.  This is just 
> 1242 stuff. 
> 
> Jim McQuaid 
> NetIQ Raleigh 
> 
> -----Original Message----- 
> From: [email-masked] [mailto:[email-masked]]On Behalf Of 
> Perser, Jerry 
> Sent: Wednesday, November 12, 2003 6:45 PM 
> To: [email-masked]' 
> Subject: RE: [bmwg] IPSec Terminology Question: Encryption/Decryption 
> Thr oughput 
> 
> 
> Tim, 
> 
> Before we can agree, I like to get some basic terminology correct: 
> 
> - Throughput is measured at the ingress of the DUT/SUT 
> - Forwarding Rate is measured at the egress of the DUT/SUT 
> - Layer 2 measurements are in frames 
> - Layer 3 measurements are in packets 
> 
> If anyone disagrees with these definitions, please point to 
> the correct 
> definition document.  I bring this up because I don't think 
> we are talking 
> apples to apples here. 
> 
> Jerry. 
> 
> 
> > -----Original Message----- 
> > From: Tim Van Herck [mailto:[email-masked]] 
> > Sent: Wednesday, November 12, 2003 3:17 PM 
> > To: Perser, Jerry 
> > Cc: [email-masked]' 
> > Subject: Re: [bmwg] IPSec Terminology Question: 
> Encryption/Decryption 
> > Throughput 
> > 
> > 
> >     Jerry, 
> > 
> > Let me restate that. We will define a tput test with two 
> > ipsec devices 
> > in a back to back setup on which we only measure at ingress 
> ports and 
> > may report the projected egress rate/Mbps. 
> > 
> > In other scenario's we can use the defined framesizes to 
> > measure rates. 
> > In all cases the definition of a rate doesn't change, just 
> > the framesize 
> >   used to calculate the throughput changes. Agree ? 
> > 
> > See section 9.1 
> > 
> >     Tim- 
> > 
> > > If you are going to measure at the ingress AND egress, then 
> > you will need 
> > > two different definitions for the two metrics. 
> > > 
> > > Jerry. 
> > > 
> > > 
> > >>-----Original Message----- 
> > >>From: Tim Van Herck [mailto:[email-masked]] 
> > >>Sent: Wednesday, November 12, 2003 10:46 AM 
> > >>To: Player, Timmons 
> > >>Cc: [email-masked]' 
> > >>Subject: Re: [bmwg] IPSec Terminology Question: 
> > Encryption/Decryption 
> > >>Throughput 
> > >> 
> > >> 
> > >>  Timmons, 
> > >> 
> > >>Like Michele already mentioned this will be addresses in 
> > the meth I-D 
> > >>but to answer the question, we will indeed document that you can 
> > >>represent throughput in four ways : 
> > >> 
> > >>- measured at ingress in pps (clear traffic) 
> > >>- measured at ingress in Mbps (clear traffic) 
> > >>- measured at egress in pps (crypto traffic) 
> > >>- measured at egress in Mbps (crypto traffic & using crypto 
> > pkt size) 
> > >> 
> > >>  Tim- 
> > >> 
> > >> 
> > >>>When performing an IPSec Encryption/Decryption Throughput 
> > >> 
> > >>the draft isn't 
> > >> 
> > >>>clear as to which side of the DUT pps or Mbps should be 
> > >> 
> > >>measured.  I'm 
> > >> 
> > >>>pretty sure that Mbps will be different for ingress and 
> > >> 
> > >>egress.  And pps 
> > >> 
> > >>>could be different for ingress and egress depending on the 
> > >> 
> > >>size of the input 
> > >> 
> > >>>packet, at least for the encryption throughput case. 
> > >> 
> > >> 
> > >> 
> > >>_______________________________________________ 
> > >>bmwg mailing list 
> > >>[email-masked] 
> > >>https://www1.ietf.org/mailman/listinfo/bmwg 
> > >> 
> > > 
> > > 
> > > _______________________________________________ 
> > > bmwg mailing list 
> > > [email-masked] 
> > > https://www1.ietf.org/mailman/listinfo/bmwg 
> > 
> 
> _______________________________________________ 
> bmwg mailing list 
> [email-masked] 
> https://www1.ietf.org/mailman/listinfo/bmwg 
> 
_______________________________________________ 
bmwg mailing list 
[email-masked] 
https://www1.ietf.org/mailman/listinfo/bmwg 

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: Tim Van Herck <[email-masked]>
Date: Fri, 14 Nov 2003 01:04:35 -0800

Jerry,
> Let me restate the problem assuming the loss detection happens at both the

Loss detection happens on the egress side but throughput as not a metric 
that is specific to ingress or egress, I would say.

> Does the measurement Units follow RFC1242 or RFC2285?

I would say we stick to the definition of throughput as stated in 
RFC1242 but remember that this can happen in two distinct topologies 
which will be described in detail in the methodology draft.

For IPSec Tunnel Throughput this would be the throughput of an SUT 
consisting of two identical back-to-back connected IPSec devices.

	Tim-

---

From: Tim Van Herck <[email-masked]>
Date: Fri, 14 Nov 2003 01:04:35 -0800

Jerry,

	Tim-