Subject: Re: [rmcat] Send timestamps

From: "Michael Ramalho (mramalho)" <[email-masked]>
Date: Wed, 5 Dec 2012 19:59:34 +0000

All,

Stephan Asked: "Do you see any problems with using an RTP header extension with absolute send timestamps?" (with send timestamps relative to the actual send time and not the capture timestamps as the normal RTP timestamp unit represents).

Not only don't I see any issue with absolute  timestamps, but I note that the processing of inter-departure times at the receiver (i.e., the resulting measurements being an inter-ARRIVAL time measurement) is often brittle.

That is, there are LOTS of (application-end to application-end) transport related things that result in the transport time of A GIVEN packet NOT to be  nominal. By "not nominal" I mean statistically UNEXPECTED from what you would think should occur if the measurement sample was created by what was assumed to be an underlying Normal, Gaussian process.

Things like software firewalls burping when doing some non-forwarding related processing (e.g., garbage collection), OSs adding strange delays after application-layer timestamps have been applied to the headers - not to mention the occasional burps in normal/expected Internet transport. The net result is that a pdf created from a histogram of individual packet delays would be one-sided and pretty different from a normal distribution. Why process these data samples as IF they were created by a Gaussian process - when you KNOW that they are not?

A SINGLE packet "outlier delay" not being "nominal" effects TWO bad (inter-arrival time) measurement data points. And most of the processing I have seen proposed on such data (e.g., Kalman filtering) is in the form of Euclidean processing in which the ERROR in the measurement is SQUARED before further processing. This effectively BIASES the very control mechanism you want to use with outlier data!

So, like Stephan, I would prefer to have the ability to have absolute timestamps in the header extension - as I desire to do some non-linear outlier filtering BEFORE traditional signal processing is applied from the output of my measurements. IMHO we ought not to be codifying an inter-arrival timestamp in the header when you could generate (if you wanted) inter-arrival measurements from the absolute quantities.

Michael Ramalho

From: [email-masked] [mailto:[email-masked]] On Behalf Of Sergio Garcia Murillo
Sent: Wednesday, December 05, 2012 4:28 AM
To: [email-masked]
Subject: Re: [rmcat] Send timestamps

El 05/12/2012 9:47, Stefan Holmer escribió:

On Tue, Dec 4, 2012 at 6:27 PM, John Leslie <[email-masked]<mailto:[email-masked]>> wrote:
Stefan Holmer <[email-masked]<mailto:[email-masked]>> wrote:
>
> Thinking further about this, there seems to be a couple of benefits with
> defining a new, absolute, send timestamp header extension. If a relay
> approach to conference servers are used, the relaying server can simply
> rewrite the send timestamp of a packet to correspond to the time it was
> relayed. Doing so the receiver can jointly process all streams being
> received from the relay server, even though they were initially captured
> and sent by different senders (with different clocks).
>
> With a send timestamp offset, accomplishing the same thing would require
> rewriting both the timestamp offset and the NTP time in the corresponding
> RTCP SR reports. And to keep video synchronized with audio the RTCP packet
> for the audio streams must also be rewritten in a similar way. This may not
> even be possible if the audio stream is relayed through a different server.
>
> Do you see any problems with using an RTP header extension with absolute
> send timestamps? Are there any benefits with the send timestamp offset
> header extension I might be missing?
   I must admit to failing to understand what problem you think this would
solve. :^(

I might have gone into the details too fast above, sorry about that.

I'm specifically thinking of the problem of measuring the changes in one-way delay by calculating the relative inter-arrival time. In http://tools.ietf.org/html/draft-alvestrand-rtcweb-congestion-03 we calculate this as

d(i) = t(i) - t(i-1) - (T(i) - T(i-1)),

where t(i) is the receive-time and T(i) is the send-time of a packet. The send-time can be stamped on the packet in different ways, either with an offset relative to the RTP timestamp or as an absolute send time.

With a relaying conference server a receiving client may receive several streams, originating from different sending clients. As all of those streams have been relayed by the same conference server we may want to use all of those streams to estimate bandwidth between the server and the receiving client. When doing that, the problems described in my initial e-mail arise.

Hi Steffan,

Not considering packet losses, to calculate the t(i) reception time of a frame, wouldn't we just need the timestamp in the last RTP packet of a frame? I am still struggling to understand the draft to implement it in my media server, so maybe my suggestion is quite dumb, but, if we had absolute timestamps for each packet, could we improve the algorithm to work on a per rtp packet base instead of a per frame?  Something like:

   At the receiving side we are observing incoming packets, where each

   packet has an absolute sent timestamp P(i).

   Each packet is assigned a receive time p(i), which corresponds to the

   time at which the packet has been received.  A packet is delayed

   relative to its predecessor if p(i)-p(i-1)>P(i)-P(i-1), i.e., if the

   arrival time difference is larger than the timestamp difference.

   We define the (relative) inter-arrival time, d(i) as

     d(i) = p(i)-p(i-1)-(P(i)-P(i-1))

Best regards
Sergio

---

From: Stefan Holmer <[email-masked]>
Date: Thu, 6 Dec 2012 11:26:28 +0100

On Wed, Dec 5, 2012 at 8:59 PM, Michael Ramalho (mramalho) <
[email-masked]> wrote:

>  All,****
>
> ** **
>
> Stephan Asked: “Do you see any problems with using an RTP header
> extension with absolute send timestamps?” (with send timestamps relative
> to the actual send time and not the capture timestamps as the normal RTP
> timestamp unit represents).****
>
> ** **
>
> Not only don’t I see any issue with absolute  timestamps, but I note that
> the processing of inter-departure times at the receiver (i.e., the
> resulting measurements being an inter-ARRIVAL time measurement) is often
> brittle.****
>
> ** **
>
> That is, there are LOTS of (application-end to application-end) transport
> related things that result in the transport time of A GIVEN packet NOT to
> be  nominal. By “not nominal” I mean statistically UNEXPECTED from what you
> would think should occur if the measurement sample was created by what was
> assumed to be an underlying Normal, Gaussian process.****
>
> ** **
>
> Things like software firewalls burping when doing some non-forwarding
> related processing (e.g., garbage collection), OSs adding strange delays
> after application-layer timestamps have been applied to the headers – not
> to mention the occasional burps in normal/expected Internet transport. The
> net result is that a pdf created from a histogram of individual packet
> delays would be one-sided and pretty different from a normal distribution.
> Why process these data samples as IF they were created by a Gaussian
> process – when you KNOW that they are not?****
>
> ** **
>
> A SINGLE packet “outlier delay” not being “nominal” effects TWO bad
> (inter-arrival time) measurement data points. And most of the processing I
> have seen proposed on such data (e.g., Kalman filtering) is in the form of
> Euclidean processing in which the ERROR in the measurement is SQUARED
> before further processing. This effectively BIASES the very control
> mechanism you want to use with outlier data!****
>
> ** **
>
> So, like Stephan, I would prefer to have the ability to have absolute
> timestamps in the header extension – as I desire to do some non-linear
> outlier filtering BEFORE traditional signal processing is applied from the
> output of my measurements. IMHO we ought not to be codifying an
> inter-arrival timestamp in the header when you could generate (if you
> wanted) inter-arrival measurements from the absolute quantities.
>

I'm not sure we are talking about the same thing here. RFC 5450 defines an
RTP header extension which allows you to specify how much earlier or later
than the accompanying RTP timestamp a packet was sent. Adding the offset to
the RTP timestamp gives you the absolute packet send-time in the RTP
timestamp domain. So as far as I can see that should be enough for what you
want to do.

The problem arises when you want to jointly process packets from different
sending clients which have been relayed through the same conference server
because the RTP timestamps will be in different time domains.

> ****
>
> ** **
>
> Michael Ramalho****
>
> ** **
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf
> Of *Sergio Garcia Murillo
> *Sent:* Wednesday, December 05, 2012 4:28 AM
> *To:* [email-masked]
> *Subject:* Re: [rmcat] Send timestamps****
>
> ** **
>
> El 05/12/2012 9:47, Stefan Holmer escribió:****
>
>  ** **
>
> ** **
>
> On Tue, Dec 4, 2012 at 6:27 PM, John Leslie <[email-masked]> wrote:****
>
> Stefan Holmer <[email-masked]> wrote:
> >
> > header extension I might be missing?****
>
> solve. :^(****
> ** **
> I might have gone into the details too fast above, sorry about that.****
>
> ** **
>
> I'm specifically thinking of the problem of measuring the changes in
> one-way delay by calculating the relative inter-arrival time. In
> http://tools.ietf.org/html/draft-alvestrand-rtcweb-congestion-03 we
> calculate this as****
>
> ** **
>
> d(i) = t(i) - t(i-1) - (T(i) - T(i-1)),****
>
> ** **
>
> where t(i) is the receive-time and T(i) is the send-time of a packet. The
> send-time can be stamped on the packet in different ways, either with an
> offset relative to the RTP timestamp or as an absolute send time.****
>
> ** **
>
> With a relaying conference server a receiving client may receive several
> streams, originating from different sending clients. As all of those
> streams have been relayed by the same conference server we may want to use
> all of those streams to estimate bandwidth between the server and the
> receiving client. When doing that, the problems described in my initial
> e-mail arise.****
>
> ** **
>
> ** **
>
> Hi Steffan,
>
> Not considering packet losses, to calculate the t(i) reception time of a
> frame, wouldn't we just need the timestamp in the last RTP packet of a
> frame?
>

Yes.

> I am still struggling to understand the draft to implement it in my media
> server, so maybe my suggestion is quite dumb, but, if we had absolute
> timestamps for each packet, could we improve the algorithm to work on a per
> rtp packet base instead of a per frame?  Something like:
>

Sure, but that's possible with send timestamp offsets relative to the rtp
timestamp as well, as already proposed.

>
>
> ****
>
>    At the receiving side we are observing incoming packets, where each ****
>
>    packet has an absolute sent timestamp P(i).****
>
> ** **
>
>    Each packet is assigned a receive time p(i), which corresponds to the****
>
>    time at which the packet has been received.  A packet is delayed ****
>
>    relative to its predecessor if p(i)-p(i-1)>P(i)-P(i-1), i.e., if the ****
>
>    arrival time difference is larger than the timestamp difference.****
>
> ** **
>
>    We define the (relative) inter-arrival time, d(i) as****
>
> ** **
>
>      d(i) = p(i)-p(i-1)-(P(i)-P(i-1))****
>
>
>
> Best regards
> Sergio****
>