Subject: RE: [bmwg] dsmterm-05 - Forwarding Congestion

From: "John Dawson" <[email-masked]>
Date: Thu, 6 Mar 2003 12:28:16 -0800

My assumption is that the traffic generator is going to be running properly
and not intentionally injecting errors.  Therefore if the traffic generator
said it sent x packets with no internal errors or late-collisions, then I
believe it.  My assumption is that the cited clause applies to errors that
the transmitter can sense.

However, we can not say that successfully transmitted and successfully
received are the same things.

Imagine a DUT that has a lot of noise coupling into the PHY receiver.  As
more packets are processed the noise floor increases and causes more inbound
packets to be corrupted.  Even though the test is run at 100% intended load,
the DUT could be corrupting untold quantities of packets.  We would forgive
the poor product design and report the test as passing at 100% intended
load?  The only way to catch that anything suspicious was occurring is by
enforcing the rule that we should be reporting results based upon offered
load -- which would now be something less than 100%.  If our corruption was
only .1%, then removing anything the DUT declares as an error would lead to
a 99.9% throughput score.

Now let's take the same imaginary DUT, that is corrupting 0.1%, and test it
at 90% load.  Maybe the DUT is only corrupting 0.09% at this load rate.
Would we still consider it as passing also?

The entire purpose of throughput testing was to measure the data rate at
which NO Loss occurs.  If we start allowing certain loss conditions, then
where do we draw the line.  I certainly would not advocate allowing the DUT
to declare the pass/fail criteria.

Maybe we have to say that zero loss must actually allow for some defined
allowance for BER (i.e.. anything less than 1e-08 or 1e-10 is considered a
random bit error and who cares why it was lost).  Assuming some allowance
for BER doesn't change the process for measuring offered load, it changes
the process for measuring loss.  Providing a quantified BER acceptance
clause is a lot different than saying, look at the DUT and forgive anything
that appears to be a physical or data link error.

I go back to my original proposition; that I don't believe the RFCs have a
done a good job of defining the process to measure offered load, and at some
point measurements requiring absolutely no loss will become a practical
impossibility.

John

-----Original Message-----
From: Perser, Jerry [mailto:[email-masked]] 
Sent: Thursday, March 06, 2003 11:21 AM
To: Dawson, John [BLVWA:382:EXCH]; [email-masked]
Subject: RE: [bmwg] dsmterm-05 - Forwarding Congestion

John,

I'll start with a quote from RFC2285, section 3.5.2 Offered load:

      Frames which are not successfully transmitted by an external
      source of traffic to a DUT/SUT MUST NOT be counted as transmitted
      frames when measuring forwarding rates.

How do you define "successfully transmitted"?

My opinion is frames that has a CRC errors, Dribble Bit Errors, Alignment
Errors, or are either Oversize or Undersize lengths (RFC2889 section 5.9.3),
"are not successfully transmitted ... to a DUT/SUT".  The Data link layer
SHOULD filter these frames such that they never appear at the egress of the
DUT/SUT.

Jerry.

-----Original Message-----
From: John Dawson [mailto:[email-masked]]
Sent: Thursday, March 06, 2003 10:42 AM
To: [email-masked]
Subject: RE: [bmwg] dsmterm-05 - Forwarding Congestion

> If the packet is corrupted going into the ingress of the DUT/SUT, 
> it is observed and not counted as a valid offered packet. 

Are you saying that offered load is now dependent upon whether the DUT
detects an error with the packet?  That seems like a substantial change in
methodology -- I use the word "seems" as I don't think the RFCs have
agreement on how offered load is actually to be measured.  Previously the
entire measurement process was independent upon the DUT's view of the world
-- we could measure everything from an external perspective.  The above
statement would be a substantial departure from that philosophy.

I would say that the group has avoided problems with bit error rate for far
too long.  I argue that some measure of protection is already warranted.
For instance, including an automatic retry for a failing test case if the
measured loss is less than the expected BER for the transmission media.

Although current tests (with an occasional retry) do work for 30 to 60
seconds without significant CRC disruptions, is that going to hold up as we
move to systems with many copper gigabit ports?  Is throughput going to
become a meaningless measure if the number of offered bits exceeds practical
BER limits?

John 
-----Original Message----- 
From: Perser, Jerry [mailto:[email-masked]] 
Sent: Thursday, March 06, 2003 9:45 AM 
To: Newman, David; Barbara Denny 
Cc: [email-masked] 
Subject: RE: [bmwg] dsmterm-05 - Forwarding Congestion 

Barbara, 
This I-D as well as its predecessors (RFC1242, RFC2285, RFC2432, RFC2647) 
have ignored physical layer errors.  The focus has been to benchmark the 
device.  Physical errors can affect the metrics, but in practice don't.  In 
RFC2554 or RFC2889 test where physical layer errors were present, I have 
corrected the error and rerun the tests. 
Another point is that we (the test operator) externally observe the metrics 
at the ingress and egress of the DUT/SUT.  If the packet is corrupted going 
into the ingress of the DUT/SUT, it is observed and not counted as a valid 
offered packet. 
If "a packet gets dropped due to buffer or queue limits being exhausted", 
weather it the IP layer or lower layers, then is falls under Forwarding 
Congestion.  If 'Good in' > 'Good Out', the DUT/SUT filtered the packet 
based on congestion, not on errors. 
Jerry. 
> -----Original Message----- 
> From: David Newman [mailto:[email-masked]] 
> Sent: Thursday, March 06, 2003 9:01 AM 
> To: Barbara Denny 
> Cc: [email-masked] 
> Subject: RE: [bmwg] dsmterm-05 - Forwarding Congestion 
> 
> 
> 
> On Thu, 6 Mar 2003, Barbara Denny wrote: 
> 
> > I sent David some more info but maybe I should 
> > at least mention to the group that this kind of 
> > loss will look the same to IP.  You lost a packet. 
> > At IP, how will you know if it was due to congestion 
> > or a problem at the link layer due to interference, 
> > etc? Are you requiring that lower layers notify 
> > IP a packet gets dropped due to buffer or queue 
> > limits being exhausted? 
> 
> Not at all. One of OSI's few redeeming features was layer 
> separation -- 
> one variable at a time is a very good thing indeed. 
> 
> Let's begin by agreeing that loss induced by events at other 
> layers occurs 
> all the time, and some of these events could affect the sole variable 
> under observation here -- the DS field in the IP header. 
> Examples include 
> LOF in Sonet, backoff algorithm timeouts in half-duplex 
> Ethernet, and RF 
> issues in wireless. 
> 
> If I understand your question correctly, it is essentially 
> "aren't these 
> cases where loss exists without congestion?" and of course 
> the answer is 
> yes. 
> 
> Still, this issue is orthogonal to the definition at hand. 
> Let's look at 
> the definition of forwarding congestion again: 
> 
> "A condition in which one or more egress interfaces are offered more 
> packets than are forwarded." 
> 
> In the cases above, the egress interfaces are offered too FEW packets, 
> not too many. Given this underfeeding, we would expect the egress 
> interfaces to be able to forward all packets without loss. 
> 
> My statement was that loss on the egress interfaces ALWAYS indicates 
> forwarding congestion. I don't believe anything we've discussed here 
> contradicts that statement. I could be wrong. If I've missed something 
> please let me know -- especially if there is a case we've 
> missed where 
> loss on egress != forwarding congestion. Thanks. 
> 
> dn 
> 
> ps. By the way, the issue of extraneous variables isn't 
> addressed in other 
> bmwg documents either. I could test a switch or router and 
> get throughput 
> of 0 bit/s by unplugging a cable, yet still conform to 2889 
> or 2544.  On 
> the general question of whether we should make all efforts to 
> factor out 
> extraneous variables like this, I think we're in complete agreement. 
> 
> 
> 
> _______________________________________________ 
> bmwg mailing list 
> [email-masked] 
> https://www1.ietf.org/mailman/listinfo/bmwg 
> 
_______________________________________________ 
bmwg mailing list 
[email-masked] 
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: David Newman <[email-masked]>
Date: Thu, 6 Mar 2003 14:14:14 -0800 (PST)

> point measurements requiring absolutely no loss will become a practical

Questions: 

1. What specific changes, if any, would you recommend in the dsterm ID?

2. If we add a BER acceptance clause, would we also need to cover all
other L1 and L2 events for all other media that could contaminate an L3
experiment? Is it even possible to put together a comprehensive list?

While I'd agree on the issues at hand, I'm less clear on the remedy.

dn

---

From: David Newman <[email-masked]>
Date: Thu, 6 Mar 2003 14:14:14 -0800 (PST)

Questions: 

dn