Subject: RE: [bmwg] Terminology Question: Throughput

From: Jim McQuaid <[email-masked]>
Date: Thu, 13 Nov 2003 19:25:58 -0000

No.  You cannot measure throughput without measuring ingress and egress.
The definition of throughput perhaps allows some ambiguity.  However, how
would one detect "dropped" if only looking at ingress?  Likewise, the
definition of frame loss rate clearly talks about forwarded (egress) frames;
the 'percentage' can only be calculated if you have both ingress and egress.

Finally, the definition of the throughput procedure is extremely clear about
HOW to measure throughput: by counting ingress and egress and comparing it.
Test equipment implementing this test did exactly that.  It is necessary to
achieve a sufficently long test duration to overcome buffering effects,
hence the reference in 'frame loss rate' to "steady state."

You said that your issue is that 'throughput' cannot be both an ingress and
egress metric.  In one sense, that's true.  It's a metric about throughput,
namely the relationship of inputs to outputs and it requires two
simultaneous measurements to be computed.

cheers,

Jim McQuaid
NetIQ Raleigh

------------------------------------------
1242 definition of throughput:
"The maximum rate at which none of the offered frames are dropped by the
device."

1242 definition of frame loss rate:
"Percentage of frames that should have been forwarded by a network device
under steady state (constant)load that were not forwarded due to lack of
resources."

1944/2544 definition of throughput benchmark procedure:
 'Procedure:  Send a specific number of frames at a specific rate through
the DUT and then count the frames that are transmitted by the DUT. '

-----Original Message-----
From: Perser, Jerry [mailto:[email-masked]]
Sent: Thursday, November 13, 2003 12:38 PM
To: Jim McQuaid; [email-masked]'
Subject: RE: [bmwg] Terminology Question: Throughput

Jim,

The definition leads one to believe that it is an ingress method.  "The
maximum rate at which none of the offered frames are dropped by the device"
[RFC1242] mentions offered frames, not forwarded frames.  Also the
measurement units are in N-octet input frames per second.  IMHO, ingress and
input are the same.

My issue is that throughput can not both be an ingress metric AND an egress
metric.  In any DUT/SUT that buffers (store and forward devices) the maximum
zero loss rate at the ingress will be higher than the egress.  Vendors will
use this ambiguity to engage in "specsmanship".

Jerry.

> -----Original Message-----
> From: Jim McQuaid [mailto:[email-masked]]
> Sent: Thursday, November 13, 2003 5:52 AM
> To: 'Perser, Jerry'; [email-masked]'
> Subject: RE: [bmwg] Terminology Question: Throughput
> 
> 
> I think the concept that throughput can be measured at the ingress of
> anything is incorrect.  Throughput is the ability of a DUT to 
> forward 100%
> of the offered load.  The definition more or less requires 
> measurement on
> both the ingress and egress and a comparison.  This is just 
> 1242 stuff.
> 
> Jim McQuaid
> NetIQ Raleigh
> 
> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of
> Perser, Jerry
> Sent: Wednesday, November 12, 2003 6:45 PM
> To: [email-masked]'
> Subject: RE: [bmwg] IPSec Terminology Question: Encryption/Decryption
> Thr oughput
> 
> 
> Tim,
> 
> Before we can agree, I like to get some basic terminology correct:
> 
> - Throughput is measured at the ingress of the DUT/SUT
> - Forwarding Rate is measured at the egress of the DUT/SUT
> - Layer 2 measurements are in frames
> - Layer 3 measurements are in packets
> 
> If anyone disagrees with these definitions, please point to 
> the correct
> definition document.  I bring this up because I don't think 
> we are talking
> apples to apples here.
> 
> Jerry.
> 
> 
> > -----Original Message-----
> > From: Tim Van Herck [mailto:[email-masked]]
> > Sent: Wednesday, November 12, 2003 3:17 PM
> > To: Perser, Jerry
> > Cc: [email-masked]'
> > Subject: Re: [bmwg] IPSec Terminology Question: 
> Encryption/Decryption
> > Throughput
> > 
> > 
> > 	Jerry,
> > 
> > Let me restate that. We will define a tput test with two 
> > ipsec devices 
> > in a back to back setup on which we only measure at ingress 
> ports and 
> > may report the projected egress rate/Mbps.
> > 
> > In other scenario's we can use the defined framesizes to 
> > measure rates. 
> > In all cases the definition of a rate doesn't change, just 
> > the framesize 
> >   used to calculate the throughput changes. Agree ?
> > 
> > See section 9.1
> > 
> > 	Tim-
> > 
> > > If you are going to measure at the ingress AND egress, then 
> > you will need
> > > two different definitions for the two metrics.
> > > 
> > > Jerry.
> > > 
> > > 
> > >>-----Original Message-----
> > >>From: Tim Van Herck [mailto:[email-masked]]
> > >>Sent: Wednesday, November 12, 2003 10:46 AM
> > >>To: Player, Timmons
> > >>Cc: [email-masked]'
> > >>Subject: Re: [bmwg] IPSec Terminology Question: 
> > Encryption/Decryption
> > >>Throughput
> > >>
> > >>
> > >>	Timmons,
> > >>
> > >>Like Michele already mentioned this will be addresses in 
> > the meth I-D 
> > >>but to answer the question, we will indeed document that you can 
> > >>represent throughput in four ways :
> > >>
> > >>- measured at ingress in pps (clear traffic)
> > >>- measured at ingress in Mbps (clear traffic)
> > >>- measured at egress in pps (crypto traffic)
> > >>- measured at egress in Mbps (crypto traffic & using crypto 
> > pkt size)
> > >>
> > >>	Tim-
> > >>
> > >>
> > >>>When performing an IPSec Encryption/Decryption Throughput 
> > >>
> > >>the draft isn't
> > >>
> > >>>clear as to which side of the DUT pps or Mbps should be 
> > >>
> > >>measured.  I'm
> > >>
> > >>>pretty sure that Mbps will be different for ingress and 
> > >>
> > >>egress.  And pps
> > >>
> > >>>could be different for ingress and egress depending on the 
> > >>
> > >>size of the input
> > >>
> > >>>packet, at least for the encryption throughput case.
> > >>
> > >>
> > >>
> > >>_______________________________________________
> > >>bmwg mailing list
> > >>[email-masked]
> > >>https://www1.ietf.org/mailman/listinfo/bmwg
> > >>
> > > 
> > > 
> > > _______________________________________________
> > > bmwg mailing list
> > > [email-masked]
> > > https://www1.ietf.org/mailman/listinfo/bmwg
> > 
> 
> _______________________________________________
> bmwg mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/bmwg
> 

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: Tim Van Herck <[email-masked]>
Date: Fri, 14 Nov 2003 00:51:13 -0800

I agree with Jim here. As far as I understand throughput is a comparison 
between 'ingress' offered load vs. 'egress' receive rate.

Only diff in our case is that we will need to define (as is done) a 
tunnel throughput which is a b2b throughput of an SUT which ipsec 
employed. Furthermore there is also a need to look at encryption and 
decryption tput since they may not be equal. For this we will need to 
rely on ipsec aware testequipment.

	Tim-

---

From: Tim Van Herck <[email-masked]>
Date: Fri, 14 Nov 2003 00:51:13 -0800

Tim-