Subject: RE: [ippm] IP Delay variation selection function

From: "Geib, Ruediger" <[email-masked]>
Date: Mon, 28 May 2001 10:24:39 +0200

Pasztor and Veitch presented at PAM 2001 a document called "A precision infrastructure for active probing". There they present bottleneck bandwidth measurement results based on delay differences of packets sent back to back. The method is based on Lai and Bakers "Measuring link bandwidths using a deterministic model of packet delay". 

The basic idea is to send two back to back packets and measure the delay difference caused by the bottleneck. That's an IPDV measurement.

There may be other purposes for measurements at link speed or buffer time scale measurement. Pairs of packets with a sending distance of back to back up to less than say 200 ms seem to be a reasonable measurement to me. That may not quite be what was meant by a "selection function". I think it's a good idea to cover useful evaluations of IPDV measurements by the standard to be written, so this method of IPDV measurements should be in there.

Looking at a selction function reasonable for applications may be to check the sending properties of the application first. If the application is creating a streamed packet flow, the function to select ipdv measurements should correspond to the minmum interval of the packets sent by the application.  

Regards, Rüdiger

> -----Original Message-----
> From: Patricia K. Maier [mailto:[email-masked]]
> Sent: Thursday, May 24, 2001 3:08 PM
> To: [email-masked]
> Subject: [ippm] IP Delay variation selection function
> 
> 
> I have been looking for some documented criteria on what 
> constitutes a "good" selection function to measure the delay
> variation. I have not seen anything in the mail archives 
> that offers any guidance. An intuitively appealing metric
> is some measure of the "width" of the distribution of the
> packet delay. Then the resulting selection function would be 
> equivalent to choosing an upper and lower percentile from
> the population of packets under consideration between two
> selected measurement points.  Specifying F effectively to
> choose a range would provide a consistent way to obtain the
> delay variation and permit some meaningful comparison across 
> measurements made by different implementations.
> 
> Are there any other known contributions to this metric within 
> this group or elsewhere?
> 
> Thanks
> PK Maier
> 
> _______________________________________________
> ippm mailing list
> [email-masked]
> http://mailhost.advanced.org/mailman/listinfo/ippm
> 
_______________________________________________
ippm mailing list
[email-masked]
http://mailhost.advanced.org/mailman/listinfo/ippm

---

From: Darryl Neil Veitch <[email-masked]>
Date: Tue, 29 May 2001 00:10:20 +1000

Dear OWDPers and others,

since Geib Rüdiger mentioned our work, it seems a good time to make a few comments- I have been following the OWDP discussions with interest, and I
have been wanting to make a few points for some time now on delay and loss issues.  My comments may well be too impractical in several respects with
respect to what is feasible within the protocol as it
stands, my opologies in advance.

My essential point of view is that generality is highly desirable, and I agree strongly with  Benjamin Teitelbaum's mail of the 14th May where he
argues for general inter-departure times (IDT's) and packet sizes, as well as Andy Scherrer's comments on being able to say `type 6 stream please'
(14th May) and by the comments of many that Poisson is too restrictive.
I believe that much more information about route state than is commonly suggested can be extracted by cleverly designed probe streams, and so by
limiting oneself to `Poisson' or minor variants a great deal is missed, which is not what is needed for a protocol designed for measurement.  Simple,
light weight measurement modes should be included, but surely more advanced methods must be at least allowed, even if we may not know how to define
them yet.  Furthermore `more advanced' means more sophisticated,
which does not necessarily imply heavy computational burdens nor even too
many extra parameters.  You can define a multifractal with just one 
parameter..  

In the discussions I find a number of what could be called `statistical' points are somewhat unclear.  Some important points I think are:
-  the `Poisson streams' referred to are of course not Poisson, as they 
   are truncated, and disturbed by the packet sizes. This being the 
   case, there is no reason to hold to them religiously, especially
   since the `PASTA' principle of simple queueing systems (Poisson
   Arrivals see Time Averages), does not hold in any way for end-to-end 
   delay measurements.  Without this, Poison is simply one way to get a 
   spread of values to avoid unpleasant synchronisation.
-  While Poisson gives you a spread of sizes, 
   it should be remembered that a Poisson (or pseudo Poisson) process 
   is essentially one with only a single time-scale, 
   the average IDT, and is thus morally close to a constant 
   rate. It is therefore not capable of `exciting' the route at multiple 
   time-scales and is doomed to match high resolution with high
   average probing rate, which is too invasive, leaving us with low
   rate low resolution probing, which only tells us the grosser 
   features of what is going on.  
-  when generalisations are discussed, people talk about changing `the 
   distribution', but the inter-departure distribution does not 
   determine a sending process, it is only ONe aspect of it. What is 
   often meant implicitly in effect is IID (independent indentically
   distributed) IDT's, but again this restricts us to a very
   small corner of the probing universe, what about correlations between
   IDT's?    Thus what Benjamin was proposing (or if not, 
   what I wished he proposed) was not that general IDT's be allowed in 
   the sense of a general (IID) DIstribution (this is still very restrictive) 
   but general as in arbitary NUmbers, which could be generated by
   unrestricted mechanisms (ie more complex RAndom processes) as
   developed over time. This IS full generality.  
-  Again, `more complex' does not necessarily imply at all that they
   would be onerous to generate, and by having them random, there is
   no issue of having to transmit the full stream in advance as some 
   have been discussing, you Want it to vary, only the parameters and 
   the model need be known. Furthermore losses should be allowed, the 
   receiver can be very confident in reasonable time that a packet 
   has been lost if sequence numbers can be included in the payload, 
   as packet reordering is not So common.  I say this as it was suggested
   that sending a specification of the stream in advance was needed
   as the receiver needed to know when loss had occurred.  This does not
   make a lot of sense as the route will seriously perturb the sending
   stream anyway, indeed this is the whole point.
-  again on this point of random sending, 
   there seems to be a issue regarding being able to send Exactly the 
   same stream, by specifying seeds and so on.  Although this may have 
   its uses as a special option, it does fly in the face of the whole
   philosophy of traffic measurement and engineering as I see it: 
   traffic is modelled as random, so even if the input is fixed, the
   measured output will be random (due to cross traffic) and therefore 
   it is more coherent to use random input (indeed this is what the 
   `Poisson' models always intended to provide).  Now random input does
   not exclude deterministic features (eg packet pairs). For example you
   could have random inter-pair times, and/or random group sizes 
   (triads not pairs), and there are advantages to these approaches.
    These `deterministic' features are strictly speaking just highly
   correlated features of the random sample paths in the random view of 
   traffic, although of course it is intuitively and practically useful 
   to consider their deterministic character as well.

We think that a much more general viewpoint is need. A probing process could modulate its packet size in a completely general way (with some limits of
course), and its IDT's according to Any process one desires, of arbitrary complexity. These do not even have to be stationary. Indeed, any packet pair
based method involves a highly correllated IDT sequences which fall well outside the simple Poisson model, even though this model IS still
 stationary provided one randomises the initial condition in the right way. 

 In this connection, note that sending rate is a TIME SCALe phenomenon, in a packet pair technique, the average rate is one parameter, but locally, on
small timescales, the sending rate is at line rate over the packet pairs. In more general streams many different time scales could be inserted.
This brings us to the question of whether sending rate matters, explored (again on the 14th May) by Henk and others. I'm afraid I disagree with Henk,
the sending rate does matter because it is not only the average rate that counts (in fact one of the holy grails of active measurement is to be able
to measure whatever you want even at Very low (average) rate!!)  but the entire sending pattern, and the higher the sending rate, the more structure
can be inserted into the pattern. Having a bottleneck which is not at the access is excellent news for the sender, it means that the higher resolution
details of what is put in will survive for longer, and can therefore probe for longer. Yes the bottleneck will erase this detail, in the inter-arrival
times, but not in the delay series...

Another general point is what to do with the measurements, what metric to choose.  I agree with Steven Van den Berghe (14th again) that the full data
must be condensed, but there is no need to reduce the full richness of
the probe stream, plus the transformation of it that is received at the receiver, by a couple of numbers such as mean and variance. It is clear that
such simple metrics cannot hope to capture too much, and I have been distressed to read over the last 24 months or more the amount of energy going
into discussing exactly how to estimate a variance, with very little discussion of metrics that could be much more important.
Part of the work Attila Pasztor and I are involved in now aims at
developing new metrics based on a measurement of the full distribution of the delay series- not to match it to some classic standard, but
to extract specific, network motivated features from it (in fact our approach is not based on Lai and Baker's work, it is quite different,
although we have similar aims and of course we both wish to exploit
back to back packets).  Speaking quite generally, condensation of information is necessary, and always implies MODelling.  Parameters do not exist in
their own right, but only mean something in the context of a model, even if that model is implicit.  What we need is a route model which is rich
enough to capture important behaviour, and with parameters that correspond to a large data reduction, but without throwing out the baby with the
bathwater.
Many parameters of interest are not related to simple means in any obvious
way, the mean and variance of delay have been chosen more for their familiarity as statistics rather than as parameters of a model. 
Seeing them as model free is a symptom of this problem.  

Finally (nearly finished now), some points have been made regarding clock rate synchronisation and how this could be corrected for. We think it is
possible to do so simply and effectively without resorting to GPS, just
by improving clock software, but nothing is written up at this point. This is a key point for the practical usefulness of (some of) the more
interesting techniques. 

Darryl 

+----------------------------+--------------------------------------------+
|  Darryl Veitch             | Email:   [email-masked]              |
|  EMUlab                    | Telephone:                                 |
|  Department of Electrical  |        Direct-   +61 3 8344 9196           |
|  & Electronic Engineering, |     Inquiries-   +61 3 8344 9204           | 
|  University of Melbourne   |                                            | 
|                            | Fax:             +61 3 8344 9188           |
|  Victoria 3010             |                                            |
|  Australia                 | Web: http://www.emulab.ee.mu.oz.au/~darryl |
+----------------------------+--------------------------------------------+
_______________________________________________
ippm mailing list
[email-masked]
http://mailhost.advanced.org/mailman/listinfo/ippm

---

From: stanislav shalunov <[email-masked]>
Date: 30 May 2001 12:53:40 -0400

Darryl,

I agree with your comments that Poisson stream of packets isn't
something special (even if it's truely and really Poisson).  It's
simply a way of measuring singleton characteristics (when used with a
sufficiently low rate).  It's also what's defined in current
standards, so there's no real reason not to stick with it for the
purposes of singleton measurements.

I believe that the current form of the draft is adequate for singleton
characteristics measurements; at the same time I agree that if we can
specify more than singleton measurements it's definitely benefitial to
do so.  Measurements of distribution of the arrival times vector for
n-tuples, for example, could prove useful.

We need to communicate the packet send schedule beforehand to be able
to detect losses.  This can be done in a variety of ways; e.g., one
could simply send all the timestamps.  This would increase control
overhead by roughly 50% and comprise roughly up to 20% of total OWDP
traffic (assuming long sessions and results that are retrieved exactly
once; if results are stored at the receiver, it's roughly 50% of total
OWDP traffic).

Can we afford to just choose full generality at the expense of network
resource consumption?  Maybe.  Should we give the opportunity to have
full generality?  Probably (but it goes beyond measuring currently
defined metrics).  Should we try to compress the schedule for "typical
measurement cases"?  Absolutely.

The question essentially is, how to compress the send schedule for
those typical measurement cases.  I don't like the answer "identify
them later and add them slowly one-by-one, thusly creating 50 protocol
variations just so we have something to do in the future".  This
protocol could be put in hard-to-upgrade devices (little measurement
nodes that you could put into the back of your network closet).
Regularly changing it is a Bad Idea.

What Ben Teitelbaum proposes is a repeated fixed-intervals schedule
with intervals between repetitions distributed independently and
exponentially or in some other simple way.  (When the number of
repetitions is 1 you achieve full generality by fully specifying the
schedule in advance.)  So far, in my opinion, it's the best proposal
that goes beyond singleton metrics.  It includes little-overhead
measurements of currently defined metrics as well as many metrics that
are clearly useful.  It also includes full generality as a special
case (without a switch statement).

Are there any better answers?

-- 
Stanislav Shalunov		http://www.internet2.edu/~shalunov/

This message is designed to be viewed at 600 mph.
_______________________________________________
ippm mailing list
[email-masked]
http://mailhost.advanced.org/mailman/listinfo/ippm

---

From: Attila Pasztor <[email-masked]>
Date: Fri, 01 Jun 2001 11:27:26 +1000

Hi Stanislav,

Please excuse me if my question is not appropriate. In your mail to
Darryl you make the following statement:

> to detect losses.

Why? I am sorry, but I don't get it.

I believe that the loss detection should be based on packet serial
numbers. If it is a problem to detect when a measurement starts or when
it has ended, it can be communicated with simple control messages between
the sender and the receiver.

1. Sender sends a "Measurement starts" message
2. Receiver sends back an acknowledgement

3. If the ack is received Sender starts to send the measurement packets;
if the ack is not received within a finite time, then it resends message
#1
4. Sender continues to send packets according to his preferred timing
schedule

5. Sender sends an "End of measurement" message, including the serial
number of the last packet sent
6. Receiver sends back an acknowledgement
7. If the sender does not receive an ack within a finite time the sender
resends message #5

(Note: limit the number of resending the messages, if no ack is received
after n trials, then the measurement has failed)

8. If there are any packets missing, after a certain time receiver
assumes they are lost

I think that the process described above detects the losses reliably
(except packet delays exceeding the time outs).
Including serial numbers in the packets is needed anyway, I don't think
that losses could be detected reliably only based on the sending
schedule.

Please, explain to me, why do we need to communicate the packet send
schedule beforehand to be able to detect losses?

By the way, in RFC 2680 'A One-way Packet Loss Metric for IPPM' there is
no mentioning of communicating the send schedule.

Thanks,
Attila