Subject: Re: [AVT] [Fecframe] Timestamps issue in Draft "RTP Payload format for 1-DInterleaved Parity FEC"

From: Magnus Westerlund <[email-masked]>
Date: Fri, 12 Sep 2008 10:43:39 +0200

Ali C. Begen (abegen) skrev:
> You are correct. Note that the 1-D draft is adopting SMPTE 2022-1 spec
> and fixing the RTP issues with it. The SMPTE spec does not use the
> timestamp field at all (receivers ignore it and what the sender puts in
> that field is undefined). In our draft, we use it and set it as you
> described below.
> 
> The motivation for this change from 2733 is that the clock at the
> instant the fec packet is transmitted is not a very useful information.
> It does not help in FEC decoding process, either. Furthermore, unlike
> the source RTP packets, FEC packets don't have a synchronization issue.
> So their transmission time is not very critical (as long as they arrive
> on time for FEC decoding). 
> 
> Instead, during our discussions, we decided to set it differently. Of
> course, FEC packets identify which source packets they protect (based on
> the SN base, L, D). However, if the FEC packet is received earlier than
> the source packets it protects (might be the case), by checking the
> timestamp field, we can now know the timestamp of the earliest source
> packet that is protected by this packet (in addition to its seqnum).
> This is a useful information. Or, if that earliest source packet won't
> arrive (i.e., it is lost), we will know by when we have to recover it.
 >
> OTOH, if the source packets are received earlier than the FEC packet,
> the timestamp info would not be that useful.
> 
> We can think of it as follows as well: The FEC packet has some
> information from that earliest source packet. That source packet was
> sampled at time say, t, so, it makes sense to put the value of t in the
> timestamp field of the fec packet. 
> 
> (RFC 3550 says that "The timestamp reflects the sampling instant of the
> first octet in the RTP data packet.")

The problem with this definition is that you are completely screwing up 
the RTCP jitter calculation. Sure, RTCP jitter calculations are screwed 
up by some other RTP payload formats, like video with multiple packets 
for the same frame. However, there is a trade-off here. I am not certain 
that the timestamp of the source packet are that useful as you anyway 
have an explicit reference to the packets you will need. If the SN base 
packet hasn't arrived, then the repair packet is early. Not particular 
likely unless strange transport reordering mechanisms are in place. If 
as you say the repair packet arrives after SN base source packet, then 
you can look up the corresponding timestamp.

So in this case, I don't see how loseing the possibility use the RTCP 
jitter for something useful on the FEC stream is motivated. And if the 
FEC stream jitter measurements works decently, then they can be used to 
cover for source streams that aren't able to utilize the jitter measurement.

I think we should include AVT in RTP related discussions. I also need to 
add this bit of information to the RTP payload how to.

Cheers

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Färögatan 6                | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt

---

From: "Ali C. Begen (abegen)" <[email-masked]>
Date: Fri, 12 Sep 2008 08:56:43 -0700

Hi Magnus,

Comments are inline. 

> -----Original Message-----
> From: Magnus Westerlund [mailto:[email-masked]] 
> Sent: Friday, September 12, 2008 11:44 AM
> To: Ali C. Begen (abegen)
> Cc: Guy Bonneau; [email-masked]; IETF AVT WG
> Subject: Re: [Fecframe] Timestamps issue in Draft "RTP 
> Payload format for 1-DInterleaved Parity FEC"
> 
> > timestamp field at all (receivers ignore it and what the 
> sender puts 
> > in that field is undefined). In our draft, we use it and 
> set it as you 
> > described below.
> > 
> > So their transmission time is not very critical (as long as they 
> > arrive on time for FEC decoding).
> > 
> > course, FEC packets identify which source packets they 
> protect (based 
> > on the SN base, L, D). However, if the FEC packet is 
> received earlier 
> > than the source packets it protects (might be the case), by 
> checking 
> > the timestamp field, we can now know the timestamp of the earliest 
> > source packet that is protected by this packet (in addition 
> to its seqnum).
> > sampled at time say, t, so, it makes sense to put the value of t in 
> > the timestamp field of the fec packet.
> > 
> > (RFC 3550 says that "The timestamp reflects the sampling instant of 
> > the first octet in the RTP data packet.")
> 
> The problem with this definition is that you are completely 
> screwing up the RTCP jitter calculation. Sure, RTCP jitter 
> calculations are screwed up by some other RTP payload 
> formats, like video with multiple packets for the same frame. 

We are not touching the jitter measurement for the source packets. But, yes, our setting will disable the jitter measurement for the FEC packets.

> However, there is a trade-off here. I am not certain that the 
> timestamp of the source packet are that useful as you anyway 
> have an explicit reference to the packets you will need. If 
> the SN base packet hasn't arrived, then the repair packet is 
> early. Not particular likely unless strange transport 
> reordering mechanisms are in place. If as you say the repair 
> packet arrives after SN base source packet, then you can look 
> up the corresponding timestamp.

Agree.
 
> So in this case, I don't see how loseing the possibility use 
> the RTCP jitter for something useful on the FEC stream is 
> motivated. And if the FEC stream jitter measurements works 
> decently, then they can be used to cover for source streams 
> that aren't able to utilize the jitter measurement.

I don't think jitter measurements based on the FEC packets could be effectively used for estimating the jitter for the source packets. There are several different sending arrangements adopted for sending the FEC packets that will mess up the measurements anyway. Furthermore, the repair packets might be sent over a different network, qos class, etc.

But, why do you need something like this in the first place? We can measure the jitter in the source stream itself.

> I think we should include AVT in RTP related discussions. I 
> also need to add this bit of information to the RTP payload how to.

Agree. Thanks for the note.

-acbegen

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 15 Sep 2008 13:22:21 +0200

> I don't think jitter measurements based on the FEC packets could be effectively used for estimating the jitter for the source packets. There are several different sending arrangements adopted for s…

Okay, so you are saying that despite the FEC source intention to 
correctly timestamp data as being sent a downstream node will mess with 
this so that it can't be used reliably?

> But, why do you need something like this in the first place? We can measure the jitter in the source stream itself.

As there are a number of RTP payload formats that you can't do jitter 
measurements on, especially for video, using the FEC RTP stream for this 
would allow jitter measurements also in these cases that otherwise can't 
be covered at all with the regular measurements.

Cheers

Magnus Westerlund

---

From: Marshall Eubanks <[email-masked]>
Date: Mon, 15 Sep 2008 07:46:43 -0400

Hello;

On Sep 12, 2008, at 11:56 AM, Ali C. Begen (abegen) wrote:

> We are not touching the jitter measurement for the source packets.  
> But, yes, our setting will disable the jitter measurement for the  
> FEC packets.
>
> I don't think jitter measurements based on the FEC packets could be  
> effectively used for estimating the jitter for the source packets.  
> There are several different sending arrangements adopted for sending  
> the FEC packets that will mess up the measurements anyway.  
> Furthermore, the repair packets might be sent over a different  
> network, qos class, etc.
>

I would disagree with this. Precisely because you might send the FEC  
packets differently, it would be useful to
know their time delay & jitter. Suppose you put FEC packets in a QOS  
class that causes them to sometimes arrive too late, because of  
buffering or path differences. How would I be able to troubleshoot  
that at the receiver if they

- all have a timestamp a few 100 milliseconds or more too early (the  
lowest sequence number means a large delay) and
- will likely have jumps in this time stamp delay  due to, e.g.,  
dropped frames in the encoder ?

However, there could be FEC schemes where the original clock is not  
available (i.e., middleware FEC). So,
maybe the text should be

The timestamp MUST be set to the value of the media RTP clock at the
instant the FEC packet is transmitted, if the media RTP clock is  
avalable to
the sender of the FEC packets. If the media RTP clock is not available
to the sender of the FEC packets, the timestamp MUST be set to the  
value of a parallel RTP
clock maintained by the sender, and this RTP clock SHOULD be  
synchronized with the
media RTP clock.

Note that the SMPTE spec is silent here, so they are not a issue.

Regards
Marshall

> But, why do you need something like this in the first place? We can  
> measure the jitter in the source stream itself.

> Fecframe mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/fecframe

---

From: "Ali C. Begen (abegen)" <[email-masked]>
Date: Mon, 15 Sep 2008 07:08:48 -0700

> >> jitter for something useful on the FEC stream is motivated. And if 
> >> the FEC stream jitter measurements works decently, then 
> they can be 
> >> used to cover for source streams that aren't able to utilize the 
> >> jitter measurement.
> > 
> > I don't think jitter measurements based on the FEC packets 
> could be effectively used for estimating the jitter for the 
> source packets. There are several different sending 
> arrangements adopted for sending the FEC packets that will 
> mess up the measurements anyway. Furthermore, the repair 
> packets might be sent over a different network, qos class, etc.
> 
> Okay, so you are saying that despite the FEC source intention 
> to correctly timestamp data as being sent a downstream node 
> will mess with this so that it can't be used reliably?

Yes. The sender may timestamp the FEC packets carefully and it also knows which sending arrangement it will use to send them. However, as long as this information (how the FEC packets were sent at the sender side, e.g., it may be bursty, interleaved, etc), the receiver will not be able to make an accurate jitter measurement. 
 
> > 
> > But, why do you need something like this in the first 
> place? We can measure the jitter in the source stream itself.
> > 
> 
> As there are a number of RTP payload formats that you can't 
> do jitter measurements on, especially for video, using the 
> FEC RTP stream for this would allow jitter measurements also 
> in these cases that otherwise can't be covered at all with 
> the regular measurements.

I understand your point now. I did not think of this problem before. I also agree that it would be cool to estimate the source jitter from the FEC packets. However, per my comments above, I am not sure whether we can do it reliably.

-acbegen

---

From: David R Oran <[email-masked]>
Date: Mon, 15 Sep 2008 07:10:38 -0700

I tend to agree with Marshall's comments below. A couple of more  
things embedded.

> that at the receiver if they
In fact, if anything differs about the 5-tuple in the IP header, not  
just the DSCP, the packets might get routed differently. This of  
course inclues the case where the FEC is sent to a separate multicast  
group.

> clock maintained by the sender, and this RTP clock SHOULD be
This works for me.

DaveO.

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 15 Sep 2008 16:56:57 +0200

Marshall Eubanks skrev:
> buffering or path differences. How would I be able to troubleshoot that 
> at the receiver if they
> 
> - will likely have jumps in this time stamp delay  due to, e.g., dropped 
> frames in the encoder ?
> 
> to the sender of the FEC packets, the timestamp MUST be set to the value 
> of a parallel RTP

Marshall,

If I understand you correctly you seem to want to have two things. Both 
a media timestamp that tells one in the relation to what media is 
handled at an asy glance and the possibility to measure transport 
jitter. To me those two aren't really complementary. Using media 
timestamp means errors in the timestamp. Only if you use transmission 
timestamping will you get reliable transport jitter measurement. So 
which function is the most important here?

To certain degree the freedom of definition  do works for the sender. 
Because he knows which regime that is been used and can thus determine 
if the RTCP jitter values from the receiver can be used or not. But if 
one tries to make any receiver side use of the timestamp in this case it 
is very problematic if one thinks it is media timestamps but they in 
reality are transport timestamps. So if we really are going for 
flexibility here there need to be explicit mention that the receiver 
can't reliably detect which it is.

Cheers

Magnus Westerlund

---

From: "Ali C. Begen (abegen)" <[email-masked]>
Date: Mon, 15 Sep 2008 11:04:59 -0700

> I would disagree with this. Precisely because you might send 
> the FEC packets differently, it would be useful to know their 
> time delay & jitter. Suppose you put FEC packets in a QOS 
> class that causes them to sometimes arrive too late, because 
> of buffering or path differences. How would I be able to 
> troubleshoot that at the receiver if they

It is possible to make estimations regarding the delay of the FEC packets based on the timestamp definition in the current draft and the source block size. But, I agree it won't be accurate.

But, as I explained earlier, even if we use the RTP clock at the time of the transmission as the timestamp, it still won't help us accurately estimate the delay/jitter at the receiver side. (BTW, the initial version (draft-begen-fecframe-1d2d-parity-scheme-00.txt) had this definition. The current definition is new in the -01 version)

> - all have a timestamp a few 100 milliseconds or more too 
> early (the lowest sequence number means a large delay) and
> - will likely have jumps in this time stamp delay  due to, 
> e.g., dropped frames in the encoder ?
> 
> However, there could be FEC schemes where the original clock 
> is not available (i.e., middleware FEC). So, maybe the text should be
> 
> The timestamp MUST be set to the value of the media RTP clock 
> at the instant the FEC packet is transmitted, if the media 
> RTP clock is avalable to the sender of the FEC packets. If 
> the media RTP clock is not available to the sender of the FEC 
> packets, the timestamp MUST be set to the value of a parallel 
> RTP clock maintained by the sender, and this RTP clock SHOULD 
> be synchronized with the media RTP clock.

If the WG agrees, I am OK with this suggested text but I still don't think it will bring much to the table.

> Note that the SMPTE spec is silent here, so they are not a issue.

Indeed. They don't use it.

-acbegen