Subject: One-way delay

From: Chien-Lung Wu <[email-masked]>
Date: Thu, 2 Dec 1999 16:10:44 -0500 (EST)

Hi, all:
There  are a lot of discussions about one-way delay metrics on IPPM. Can
anybody show me or point out how to measure one-way delay? I think many
works have been done on round-trip delay. Why do we need "one-way delay"?
Thanks in advance.

Regards,

Chien-Lung

****************************************************************************
Chien-Lung Wu						[email-masked]
Graduate Student of ECE 				(O) 919-513-1894
at North Carolina State University			(H) 919-233-6724
****************************************************************************

---

From: Grotefeld Glenn-cecl03 <[email-masked]>
Date: Fri, 3 Dec 1999 07:54:18 -0600

The most prevalent reason is that certain applications, such as streaming
video and/or audio, are most influenced by one-way delay.

Glenn Grotefeld
Corporate Standards
Motorola, Inc.

~-----Original Message-----
~From: Chien-Lung Wu [mailto:[email-masked]]
~Sent: Thursday, December 02, 1999 3:11 PM
~To: [email-masked]
~Cc: [email-masked]; [email-masked]
~Subject: One-way delay
~
~
~Hi, all:
~There  are a lot of discussions about one-way delay metrics on 
~IPPM. Can
~anybody show me or point out how to measure one-way delay? I think many
~works have been done on round-trip delay. Why do we need 
~"one-way delay"?
~Thanks in advance.
~
~Regards,
~
~Chien-Lung
~
~
~***************************************************************
~*************
~Chien-Lung Wu						
[email-masked]
~Graduate Student of ECE 				(O) 919-513-1894
~at North Carolina State University			(H) 919-233-6724
~***************************************************************
~*************
~
~

---

From: [email-masked]
Date: Fri, 3 Dec 1999 11:04:51 -0500

From:  Tom Gray@MITEL on 12/03/99 11:04 AM
How  is it measured?

Jon Crowcroft <[email-masked]> on 12/03/99 10:29:05 AM

To:   "Glossbrenner, Kenneth C." <[email-masked]>
cc:   "'Chien-Lung Wu'" <[email-masked]>, [email-masked], "Annie Gravey
      (E-mail)" <[email-masked]>, "Eileen Natale (E-mail)"
      <[email-masked]>, "Neal B. Seitz (E-mail)" <[email-masked]>,
      [email-masked] (bcc: Tom Gray/Kan/Mitel)

Subject:  Re: One-way delay

you can't always substitute 1/2 RTT for one way delay, and there are
applications that require knowledge of one way delays because of
_third party_ effects (rather than interaction) - e.g. electronic trading

there are large nets in use now that don't have symmetric paths,. let
alone delays (e.g. UDLR on satellite hops with terrestrial) - some of
these are used for distribution of trading data, and might (for even
regualtory reasons) require some sort of 1 way delay info.

cheers

jon.

---

From: "Glossbrenner, Kenneth C." <[email-masked]>
Date: Fri, 3 Dec 1999 08:47:14 -0800

*** Please see below.

-----Original Message-----
From: Henk Uijterwaal (RIPE-NCC) [mailto:[email-masked]]
Sent: Friday, December 03, 1999 11:05 AM
To: [email-masked]
Subject: RE: One-way delay

On Fri, 3 Dec 1999, Glossbrenner, Kenneth C. wrote:

> IMHO there are no significant applications that care at all about one-way
> delay, they only care about round-trip delay. 

The fact that the application cannot see the difference, does not mean
that it isn't important.

*** It is not important to any applications that I understand.
*** If it's not important to the users of service, then
*** I don't care to spend a lot of extra money to measure it.
*** Synchronizing clocks with precision at opposite ends 
*** of a network is complex and expensive.
*** The exception, as you and I both note below, is not that an
*** application finds one-way delay important, but it is the need
*** to diagnose and fix a problem with excessive round-trip delay.

*** I would like to know more about the 
*** "third_party_effects" mentioned by Jon Crowcroft. I can
*** imagine the relevance if the only acknowledgement back to the
*** first party must come via a third party.

> If, for example, the round-trip delay is X, I know of no real
> application that can even tell the difference between X=X/2+X/2,
> X=X+0, and X=0+X.  Certainly users of 2-way video and audio cannot
> distinguish between these three cases. If I finish my sentence and it
> always takes at least X time before I get a reply, how can I tell
> which of the aforementioned three cases was in effect?

That is true, but now suppose that you want to improve X.  If X=X/2+X/2,
then the best strategy is to spend equal amounts of effort on both paths,
if X=X+0, then you better spend most of your efforts on reducing the delay
in the first path.

*** I agree that one-way delay is important in diagnosing and fixing
*** round-trip delay problems.  Without more details from Jon, I don't
*** agree that there are major communications applications that can 
*** see or care about one-way delay.

> There was an undocumented agreement about this point in the ITU about
> a year and a half ago.

I don't think that there is agreement on this in IPPM (and I like to see
the arguments that the ITU made to to come to this agreement sometime).

> If you can't afford to measure one-way, then two-way measurements
> divided by two are usually taken to be acceptable approximations of
> the one-way performance.

For most of the 1600 pairs of delays that we regularly measure, we see an
asymmetric behaviour, so if if you can't afford to measure one-way, then I
don't think that you should quote one-way values at all.

*** I agree that estimating one-way delay with half of
*** round-trip delay is probably too cavalier for IP network.
*** If one needs to investigate one-way delay in IP, it will
*** have to be done correctly.
*** Please pardon my experience with measurement of
*** conventional networks.

*** Ken 

Henk

----------------------------------------------------------------------------
--
Henk Uijterwaal                    Email: [email-masked]
RIPE Network Coordination Centre     WWW: http://www.ripe.net/home/henk
Singel 258                         Phone: +31.20.535-4414,  Fax -4445
1016 AB Amsterdam                   Home: +31.20.4195305
The Netherlands                   Mobile: +31.6.55861746  
----------------------------------------------------------------------------
--

The Committee (...) was unable to reach a consensus that substantial merit
was
lacking. Thus, the appeal was deemed meritorious.          (Orlando NABC
#19).

---

From: "David Smrdel" <[email-masked]>
Date: Fri, 3 Dec 1999 12:56:02 -0500

There are many applications that do care about one-way delay times.  For
instance the transfer of large amounts of data for off-site disk mirroring is a
good example.  The data is simply being transferred from point A to point B and
that delay time is critical in establishing the necessay transmit buffer sizes.
If it can be accurately determined that the propagation delay for a given
network in a given direction is relatively high, then one would avoid sending
Tetrabytes of data on this line because it could mean that a significant amount
of time may be spent in transferring this large amount of data.  This excessive
transfer  time would be avoided in leu of a faster line due to the high costs of
dedicated long distance communication lines.

You may say that if the propagation time for round-trip is high, then it is safe
to assume that both directions must be slow.  This has not been our experience.
We see that with routed and switched networks, you have no guarantee that
packets heading from point A to B will take the same path as from B to A.
While in most applications that exist on campus style networks this may be
insignificant, the same is not true when transferring data from New York to Los
Angeles and back.

"Glossbrenner, Kenneth C." <[email-masked]> on 12/03/99 09:51:52 AM

 To:      "'Chien-Lung Wu'" <[email-masked]>,            
          [email-masked]                                   
                                                              
 cc:      "Annie Gravey (E-mail)"                             
          <[email-masked]>, "Eileen       
          Natale (E-mail)" <[email-masked]>, "Neal B. Seitz  
          (E-mail)" <[email-masked]>(bcc: David       
          Smrdel/ENG/Computerm)                               

IMHO there are no significant applications that care at all about one-way
delay, they only care about round-trip delay. If, for example, the
round-trip delay is X, I know of no real application that can even tell the
difference between X=X/2+X/2, X=X+0, and X=0+X. Certainly users of 2-way
video and audio cannot distinguish between these three cases. If I finish my
sentence and it always takes at least X time before I get a reply, how can I
tell which of the aforementioned three cases was in effect? There was an
undocumented agreement about this point in the ITU about a year and a half
ago.

One-way delay might be interesting for some telemetry applications.  If an
alarm goes off, the quicker it gets to the alarm monitoring station, the
sooner mitigation can be initiated.  Of course, one-way delay is only
important to telemetry applications if the mitigation is effected
out-of-band. If the mitigation is a command that returns across the same
communication network that carried the alarm, then once again it doesn't
matter whether the round-trip communications is X=X/2+X/2, X=X+0, or X=0+X,
it still takes at least X time to effect the mitigation.

(I also note that if the out-of-band mitigation is out-of-band, e.g. rolling
a repair truck and if is X on the order of 100s or 1000s of milliseconds,
then how much does it really help to roll trucks a few milliseconds
earlier?)

The reason the ITU has steadfastly defined one-way delay metrics instead of
two-way delay metrics is that if you can afford to measure them (measurement
requires synchronized measurement devices, see e.g. X.138 for measurement
techniques), they provide additional diagnostic capabilities. If your
RT-delay is high, you will want to know why, and for IP networks in
particular, delay asymmetries are quite possible. If you can't afford to
measure one-way, then two-way measurements divided by two are usually taken
to be acceptable approximations of the one-way performance.

Ken Glossbrenner

-----Original Message-----
From: Grotefeld Glenn-cecl03 [mailto:[email-masked]]
Sent: Friday, December 03, 1999 8:54 AM
To: 'Chien-Lung Wu'; [email-masked]
Cc: [email-masked]; [email-masked]
Subject: RE: One-way delay

Glenn Grotefeld
Corporate Standards
Motorola, Inc.

---

From: "Glossbrenner, Kenneth C." <[email-masked]>
Date: Fri, 3 Dec 1999 10:35:46 -0800

I am trying to understand.

The speed from start to finish for a large file transfer is first limited by
the bit rate of the intervening transport.  In my mind that is not a measure
of the network one-way delay, but instead a measure of the capacity of the
network to support throughput.

I know that the time from start to finish for a large file transfer is
further limited because TCP waits for acknowledgements before sending more
data. Is it your contention that the one-way delay of a single packet can be
estimated by calculating how much the file transfer is constrained by this
acknowledgement process during a file transfer? Those acknowledgements are
delayed by the round-trip delay X, regardless of how X is apportioned
between the two directions, so I still fail to see how the one-way delay of
individual packets could be estimated from the speed of a file transfer.

Perhaps we are talking about two different things: network delay and time to
execute a file transfer.

Ken

-----Original Message-----
From: David Smrdel [mailto:[email-masked]]
Sent: Friday, December 03, 1999 12:56 PM
To: Glossbrenner, Kenneth C.
Cc: 'Chien-Lung Wu'; [email-masked]; Annie Gravey (E-mail); Eileen
Natale (E-mail); Neal B. Seitz (E-mail)
Subject: RE: One-way delay

There are many applications that do care about one-way delay times.  For
instance the transfer of large amounts of data for off-site disk mirroring
is a
good example.  The data is simply being transferred from point A to point B
and
that delay time is critical in establishing the necessay transmit buffer
sizes.
If it can be accurately determined that the propagation delay for a given
network in a given direction is relatively high, then one would avoid
sending
Tetrabytes of data on this line because it could mean that a significant
amount
of time may be spent in transferring this large amount of data.  This
excessive
transfer  time would be avoided in leu of a faster line due to the high
costs of
dedicated long distance communication lines.

You may say that if the propagation time for round-trip is high, then it is
safe
to assume that both directions must be slow.  This has not been our
experience.
We see that with routed and switched networks, you have no guarantee that
packets heading from point A to B will take the same path as from B to A.
While in most applications that exist on campus style networks this may be
insignificant, the same is not true when transferring data from New York to
Los
Angeles and back.

                                                              

                                                              

Ken Glossbrenner

-----Original Message-----
From: Grotefeld Glenn-cecl03 [mailto:[email-masked]]
Sent: Friday, December 03, 1999 8:54 AM
To: 'Chien-Lung Wu'; [email-masked]
Cc: [email-masked]; [email-masked]
Subject: RE: One-way delay

Glenn Grotefeld
Corporate Standards
Motorola, Inc.

---

From: "Glossbrenner, Kenneth C." <[email-masked]>
Date: Fri, 3 Dec 1999 11:48:27 -0800

Cool. I understand and agree.

-----Original Message-----
From: Stefan Savage [mailto:[email-masked]]
Sent: Friday, December 03, 1999 2:41 PM
To: 'Glossbrenner, Kenneth C.'; [email-masked]
Cc: Annie Gravey (E-mail); Neal B. Seitz (E-mail)
Subject: RE: One-way delay

If the measured path is the only causal channel then there is clearly no way
to disambiguate between one-way delay and round-trip delay.

However, it is clearly possible for an application to have more than one
causal channel.  Trivially, if you and I are both watching CNN, see NASDAQ
plummet, and then both try to unload all of our Web stocks ASAP, then I will
have an advantage if I have a shorter one-way delay to the trading floor
than you.  

Here's a more realistic example.  In an interactive multi-user network game
suppose that clients send location updates both to each other and to the
server (to improve responsiveness).  However "operations" (e.g. checking if
you killed someone) must be serialized through the server to ensure
consistent state changes.  To be concrete, lets draw the following picture,
with server S and clients A and B:

               S
              / \
             /   \    
            /     \
           A-------B

Now suppose that RTT(AB), RTT(AS), and RTT(BS) are equal.  In addition, AB
and AS have symmetric one-way delays, but DELAY(B->S) < DELAY(S->B). Client
B has an advantage because if A and B enter range of each other they will
both learn this fact at the same time, but B's operations will reach the
server first.

I'm dubious that this is a common issue today because most applications are
strictly peer-to-peer or single client, single server.  However, as more
complicated distributed applications emerge I think this difference may
matter.

- Stefan

-----Original Message-----
From: Glossbrenner, Kenneth C. [mailto:[email-masked]]
Sent: Friday, December 03, 1999 8:47 AM
To: [email-masked]
Cc: Annie Gravey (E-mail); Neal B. Seitz (E-mail)
Subject: RE: One-way delay

*** Please see below.

-----Original Message-----
From: Henk Uijterwaal (RIPE-NCC) [mailto:[email-masked]]
Sent: Friday, December 03, 1999 11:05 AM
To: [email-masked]
Subject: RE: One-way delay

*** Ken 

Henk

---

From: "David Smrdel" <[email-masked]>
Date: Fri, 3 Dec 1999 15:37:51 -0500

Ken,

In large data transfers, three things are important in determining the success
of the transfer: Data transfer rate, overhead added by intervening equipment,
and propagation delay.  I may not have been clear in my last email.  I
understand that the resultant throughput is affected by all three of these items
and that one can conceivably be able to determine progation delay by
constraining the other two parameters.  The important point that I am trying to
make is that there are those of us out there who do care about one-way delay
time.  Realizing that acks must traverse the opposite direction of data
transfer, it is still important to know what each direction's times are for we
can size our transmit buffers (not the socket buffers but our own proprietary
buffers) to ensure that we minimize retransmits and data queuing.  If we can
determine the one-way delay, it would help us in determining other parameters.

My little sockets program is used to identify the time it should take for a file
transfer.  I am surmizing that we can use this information to derive estimated
propagation delay.

Dave

"Glossbrenner, Kenneth C." <[email-masked]> on 12/03/99 01:35:46 PM

 To:      [email-masked]                                   
                                                              
 cc:      "Annie Gravey (E-mail)"                             
          <[email-masked]>, "Neal B.      
          Seitz (E-mail)" <[email-masked]>(bcc: David 
          Smrdel/ENG/Computerm)                               

Ken

-----Original Message-----
From: David Smrdel [mailto:[email-masked]]
Sent: Friday, December 03, 1999 12:56 PM
To: Glossbrenner, Kenneth C.
Cc: 'Chien-Lung Wu'; [email-masked]; Annie Gravey (E-mail); Eileen

Ken Glossbrenner

-----Original Message-----
From: Grotefeld Glenn-cecl03 [mailto:[email-masked]]
Sent: Friday, December 03, 1999 8:54 AM
To: 'Chien-Lung Wu'; [email-masked]
Cc: [email-masked]; [email-masked]
Subject: RE: One-way delay

Glenn Grotefeld
Corporate Standards
Motorola, Inc.

---

From: "Glossbrenner, Kenneth C." <[email-masked]>
Date: Fri, 3 Dec 1999 12:52:53 -0800

a) I still don't understand how you can free up any retransmission or
queuing buffers until the acknowledgement returns (in X time, regardless of
whether X=X+0, X=0+X, or X=X/2+X/2.)

b) I am not saying that no one cares about one-way delay, clearly network
engineers and maintenance people have to care.  But other than Stephan
Savage's interesting gaming application, I do not see major applications
that know or care how the total round-trip delay, X, partitions across the
two directions.

Ken

-----Original Message-----
From: David Smrdel [mailto:[email-masked]]
Sent: Friday, December 03, 1999 3:38 PM
To: Glossbrenner, Kenneth C.
Cc: [email-masked]; Annie Gravey (E-mail); Neal B. Seitz (E-mail)
Subject: RE: One-way delay

Ken,

In large data transfers, three things are important in determining the
success
of the transfer: Data transfer rate, overhead added by intervening
equipment,
and propagation delay.  I may not have been clear in my last email.  I
understand that the resultant throughput is affected by all three of these
items
and that one can conceivably be able to determine progation delay by
constraining the other two parameters.  The important point that I am trying
to
make is that there are those of us out there who do care about one-way delay
time.  Realizing that acks must traverse the opposite direction of data
transfer, it is still important to know what each direction's times are for
we
can size our transmit buffers (not the socket buffers but our own
proprietary
buffers) to ensure that we minimize retransmits and data queuing.  If we can
determine the one-way delay, it would help us in determining other
parameters.

My little sockets program is used to identify the time it should take for a
file
transfer.  I am surmizing that we can use this information to derive
estimated
propagation delay.

Dave

                                                              
 To:      [email-masked]                                   

                                                              

Ken

-----Original Message-----
From: David Smrdel [mailto:[email-masked]]
Sent: Friday, December 03, 1999 12:56 PM
To: Glossbrenner, Kenneth C.
Cc: 'Chien-Lung Wu'; [email-masked]; Annie Gravey (E-mail); Eileen

Ken Glossbrenner

-----Original Message-----
From: Grotefeld Glenn-cecl03 [mailto:[email-masked]]
Sent: Friday, December 03, 1999 8:54 AM
To: 'Chien-Lung Wu'; [email-masked]
Cc: [email-masked]; [email-masked]
Subject: RE: One-way delay

Glenn Grotefeld
Corporate Standards
Motorola, Inc.

---

From: Vern Paxson <[email-masked]>
Date: Fri, 03 Dec 1999 22:53:22 PST

There are different roles for IPPM metrics.  One is to understand how well
applications might perform along a particular network path; another is to
help understand why you get the performance you get.  I agree that for many
apps, it's round-trip delay and not one-way delay that matters.  But for
diagnosing problems, one-way delay (and, more so, one-way delay variation
and loss) is often a lot more valuable than round-trip delay - both because
routing and network path properties are often asymmetric (so one-way
measurement is better at pinning down the location of problems), and
because often for transport protocols the behavior of the different
directions (forward, data-carrying path vs. reverse acknowledgement path)
is highly relelvant.

		Vern

---

From: Vern Paxson <[email-masked]>
Date: Fri, 03 Dec 1999 23:08:35 PST

Also, one of the goals for IPPM metrics is that when possible they should
be composable.  As you can in principle derive round-trip delay from two
one-way delay measurements, this in a sense makes a one-way delay metric
more fundamental than a round-trip delay metric.  The fact that the composition
is not in practice trivial (i.e., just adding the two measurements gets you
the wrong answer, since there's some additional delay at the far end turning
around the packet) is a feature, because it highlights that a round-trip
measurement is not simply a network measurement but also measures something
about the end host.

		Vern

---

From: Stefan Savage <[email-masked]>
Date: Mon, 6 Dec 1999 07:22:26 -0800

Minor correction and shameless plug:  it IS possible to measure one-way loss
along "random" paths by leveraging the widespread deployment of TCP.  No
other remote infrastructure required.

See http://www.cs.washington.edu/homes/savage/sting/

In the future, we plan to announce similar tricks for other one-way metrics
(delay variation, re-ordering, bottleneck capacity, etc...)  I agree that
you can't do one-way delay though...

- Stefan

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]
Sent: Monday, December 06, 1999 6:44 AM
To: [email-masked]
Cc: [email-masked]; [email-masked]
Subject: RE: One-way delay

> From all of the emails that I have seen on this topic, I
> believe that the consensus is that there is a need and purpose
> for one-way delay measurement...now the original question still
> needs to be answered...what is the best way for someone to
> measure it?

The best way for someone to measure it is (briefly) to have a
measurement machine at either end of the path under study, each of
them synchronized to a known good time source.  See the RFCs and the
Internet-drafts (if there are still any for one-way delay; I somehow
doubdt it) for the nitty-gritty details.

In practice, this means that it will take quite some effort in order
to accurately measure the one-way delay and loss characteristics of
a given path, and the technique thus isn't all that amenable to
measuring these metrics for "random" paths due to the need for good
infrastructure to carry out the measurements.

Regards,

- Håvard

---

From: Robin Uyeshiro <[email-masked]>
Date: Mon, 6 Dec 1999 08:46:44 -1000

There are a few commercial testers that are able to do one-way measurements 
very accurately.  If you send me an email, I can provide a somewhat biased 
list.  I believe most use GPS to obtain higher accuracy.

Robin Uyeshiro
Adtech, Inc.

-----Original Message-----
From:	[email-masked] [SMTP:[email-masked]]
Sent:	Monday, December 06, 1999 4:44 AM
To:	[email-masked]
Cc:	[email-masked]; [email-masked]
Subject:	RE: One-way delay

> needs to be answered...what is the best way for someone to

The best way for someone to measure it is (briefly) to have a measurement 
machine at either end of the path under study, each of them synchronized to 
a known good time source.  See the RFCs and the Internet-drafts (if there 
are still any for one-way delay; I somehow doubdt it) for the nitty-gritty 
details.
In practice, this means that it will take quite some effort in order to 
accurately measure the one-way delay and loss characteristics of a given 
path, and the technique thus isn't all that amenable to measuring these 
metrics for "random" paths due to the need for good infrastructure to carry 
out the measurements.
Regards,

- Havard

---

From: KERBERENES Jacques CNET/DAC/LAN <[email-masked]>
Date: Tue, 21 Dec 1999 16:52:15 +0100

Hi every body,

> ----------
> De : 	Chien-Lung Wu[SMTP:[email-masked]]
> Date :	Thursday, December 02, 1999 10:10 PM
> A :	[email-masked]
> Cc :	[email-masked]; [email-masked]
> Objet :	One-way delay
> 
> Hi, all:
> There  are a lot of discussions about one-way delay metrics on IPPM. 
	You are right, since the beginning of the  IPPM work there has been
a lot of discussions about this topic : one-way vs round-trip. And , as for
other issues there are several schools, each of them having their own
arguments for choosing  one of these metrics. 
> Can
> anybody show me or point out how to measure one-way delay?
There are at least two current trials that are measuring one-way delay and
packet loss : Surveyor (ANS) and RIPE-NCC Test Project in Europe. I am
trying to convince France Telecom to participate to the two projects, I
think we will sign an agreement with ANS in January and with RIPE when their
new boxes will be available.
	We are also thinking to develop our own probes, and to timestamp the
probe packets at the physical layer, 
	(wire time vs host time, see the discussion in the framework
document RFC 2330)

>  I think many
	There are several motivations for the one-way measurements that are
well explained is RFCs 2679 and 2680. In particular the assymetry of the two
paths (routing and queuing) over a lot of links We know we have such traffic
patterns on some of our links. But in an other other hand, I agree that
round-trip measurements as performed, for instance by the Pinger and AMP
projects, are in fact the quantity of interest. For me (IMHO) there is no
contradiction between the two approaches. Do what we need where we need and
to obtain  the results we are looking for.

	A very interesting paper is the comparison of the results provided
by differrent tools and methodologies : Les Cottrel :
http://www.slac.standford.edu/comp/net/wan-mon/iepm-cf.html

> Chien-Lung
	Best Regards

	Jacques

> Jacques Kerberenes
> 
> e-mail : [email-masked]
> 
> Phone : +33  (0)2 96 05 29 13
> Fax :      +33  (0)2 96 05 18 52
> 
> CNET/DAC/NTR
> 2, av Pierre Marzin
> BP 40 22301 LANNION CEDEX
> FRANCE