Subject: RE: WG Last Call: New Work Proposal on Benchmarking DiffServ Mech anisms

From: "Perser, Jerry" <[email-masked]>
Date: Tue, 5 Dec 2000 10:00:40 -0800

John,
 
I have asked to be on the agenda of the 49th IETF to talk about the problems
using current metrics for testing diffserv/precedence and to talk about some
solutions.  I hope you will be able to attend.
 
I have problems with using a zero-loss metric in environment where loss is
expected.  In fact, I have an argument going on about "acceptable loss
throughput."  RFC1242 clearly defines Throughput and I would like to keep it
that way.  When you measure the Throughput of a DUT/SUT, the DUT/SUT
forwarded ALL frames that it was offered.
 
Different forces are in action when testing diffserv/precedence such as loss
induced by congestion.  The approach that we are taking is to start with a
clean slate, define these forces in action, then write a methodology.  A lot
of the work will be based on the experience of the previous RFCs.
 
Jerry Perser
  
-----Original Message-----
From: John Dawson [mailto:[email-masked]]
Sent: Monday, December 04, 2000 5:12 PM
To: Sumit Khurana; David Newman
Cc: [email-masked]; Scott Poretsky; Perser, Jerry; Tony
Bogovic; Shobha Erramilli
Subject: RE: WG Last Call: New Work Proposal on Benchmarking DiffServ Mech
anisms

Gentlemen, 

I respectfully submit the following for your consideration. 

If we are going to create a new RFC to deal with QOS and Diff-serve, can we
agree to start using a term other than Throughput?  This term currently has
a very specific and precise definition that has grown out of years of
implementation and practice.

If you start using the term Throughput in both zero-loss and
over-subscription settings, which are expected to cause measurable losses
(as is likely in diff-serve tests), then I fear you are going to confuse
everyone and render the current terminology meaningless.

Some alternate suggestions for your consideration are channel capacity,
code-point capacity, queue capacity, zero-loss forwarding rate, or
code-point forwarding rate.  I am sure that there is a wide variety of
similar terms which may convey substantially the same meaning and yet may be
far more targeted to the needs of diff-serve.

For the sake of differentiation between tests, and for the preservation of
the integrity of existing terminology, let's not try to endlessly stretch
the meaning of Throughput to include a clause or sub-clause which fits every
possible performance test.

John 

-----Original Message----- 
From: Sumit Khurana [ mailto:[email-masked]
<mailto:[email-masked]> ] 
Sent: Monday, December 04, 2000 3:34 PM 
To: David Newman 
Cc: [email-masked]; BMWG (E-mail); Scott Poretsky; 
Perser, Jerry; Tony Bogovic; Shobha Erramilli 
Subject: RE: WG Last Call: New Work Proposal on Benchmarking DiffServ 
Mechanisms 

Jerry, David, 

This is the first time that we have heard any comments on the ID 
since it was discussed at the last BMWG meeting, where there was 
initial support for the draft. May I suggest that it 
would be useful for the working group for you to enumerate the 
specific concerns that you have so that similar mistakes are avoided 
if and when the work is taken up by the working group. After all, the 
draft is the first tangible work to address the problem as opposed to the 
proposals. 

As far as the broad concern that you have raised regarding application of 
existing metrics for the measurements, that is not necessarily a bad thing 
if it captures the properties that one is trying to measure. 
Indeed we have "adapted" existing metrics for the measurements. It is 
also true, that there could  be other properties for which  one may 
require completely new  metrics and they could be incorporated into 
the methodology. Suggestions for such properties and metrics would 
be welcomed.    

We would like to suggest that the draft be taken up as the basis for the 
proposed work item. We look forward to comments on the contents of the 
draft from the mailing list. 

Thanks, 
Sumit 

On Mon, 4 Dec 2000, David Newman wrote: 

> 
> > 
> > When I look at the minutes of the last BMWG meeting, 
> >     http://www.ietf.org/proceedings/00jul/index.html
<http://www.ietf.org/proceedings/00jul/index.html>  
> > 
> > there was interest in an individually submitted (non-WG) I-D: 
> > 
> > http://www.ietf.org/internet-drafts/draft-khurana-bmwg-diffservm-00.txt
<http://www.ietf.org/internet-drafts/draft-khurana-bmwg-diffservm-00.txt>  
> > 
> > Have the proposers of the new work item regarded this work?  Are they 
> > different? The same? Are there areas of collaboration? 
> 
> Kevin and fellow bmwg'ers, 
> 
> We did indeed review the Khurana I-D. It raised for us some of the same 
> concerns that were raised by others about our previous effort -- too broad

> in scope and too liberal in its use of the term "quality of service" (a
term 
> we've deliberately avoided this time). Further, we felt the Khurana draft 
> (and, admittedly, our previous effort) took the too-simple approach of 
> applying existing metrics to a new field of measurement. We also had many 
> other concerns with the Khurana draft. We can enumerate this if the list
is 
> interested. However, we felt that it would be more constructive to submit
a 
> revised, and more focused, proposal of our own. 
> 
> Jerry Perser 
> David Newman 
> 
>