Subject: RE: [AVT] Update (-09) of Timed Tex draft (WAS: RE:Commentsondraft-ietf-avt-rtp-3gpp-timed-text-08.txt)

From: "Jose Rey" <[email-masked]>
Date: Thu, 14 Apr 2005 18:22:39 +0200

[I include Dave and Jan]

Colin,

[snip]

> I'm struggling to understand why the timestamp definition has 
> to be so complex.  The additional rules regarding Type 5 
> units don't seem to be necessary, and will likely lead to 
> confusion and implementation errors.
> 
[snip]
> > Prohibiting that a particular SIDX value is used by a 
> previous unit is 
> > not OK, since the same sample description may be used by different 
> > text samples which have different timestamps.
> 
> I don't understand your comment - the statement that a 
> "sample description conveyed in a TYPE 5 packet MUST NOT be 
> referenced by a unit with earlier timestamp" is simply that 
> type 5 units are sent before they are used.

The sentence is OK.

While understanding the consequences of this I have found a different issue.  Before going into it this is my understanding how reception should work: packets arrive at RTP client, units are extracted from payloads and assigned timestamp, also sample descriptions.  Then they are ordered in the buffer according to TS.  The server provides that the units have the required sample descriptions available at the time of decoding by assigning same timestamp.  Then we'll end up with a situation where the same sample description is in the buffer several times with different timestamp values, is this OK?    

The question is how the (real-time) decoder buffers SDs.  I don't know how this actually happens.  However, regardless of whether we change the timestamp of the SD to that of the RTP packet they are in, as proposed in my previous email or keep it as is, we have to REQUIRE a minimum buffering time for the SDs at the DECODER using dynamic SDs.  I.e., it cannot be that SDs be just discarded once they are used.  Because if an SD is present in an aggregate payload applying to several samples, this buffering time is determined by the timestamp of the last text sample in THAT payload that uses that SD; otherwise if the SD applies only to the last text sample, and since the SD is given an earlier timestamp, this is not available anymore as the last sample is to be displayed.  

For specifying the buffering time of dynamic SDs we have following possibilities:
a) new fields that encode it in the TYPE 5 unit header.  However, we cannot specify a duration for the SD because the timed text format does not understand that.
b) we REQUIRE the decoder to use the dynamic SIDX algo to know when a SD should be kept or deleted.  Is this possible?  (it is not the RTP client but the decoder that should implement it)
c) or we could avoid all this by repeating the SD for every single sample that uses it and so could SDs be discarded after use. However it could potentially be too much overhead, but gain in simplicity.

David, Jan, comments?

> You seem to be 
> suggested that they can be used before they're transmitted?
> 
> > On this regard, this discussion in the intro of section 4 
> was moved to 
> > 4.5, where the following was added:
> >
> >      Regarding sample descriptions in aggregate payloads: 
> since TYPE 5
> >      units may potentially apply to several units in the stream, a
> >      sender shall ensure that a copy of the sample description is
> >      received before the affected text sample is due.  Therefore, a
> >      copy of the sample description SHOULD be placed in the payload
> >      before the text sample it applies to.  Of course, if 
> several text
> >      samples in a payload use the same sample description, once per
> >      payload is enough.  A sender MAY choose to omit the sample
> >      description if it knows by some other means, such as payload
> >      specific feedback messages [21], that the sample 
> description has
> >      arrived at the receiver or if it employs additional transport
> >      resiliency measures (Section 5), for example.
> 
> I would suggest changing this to:
> 
> 	Correct reception of TYPE 5 units is important since 
> their contents
> 	may be referenced by several other units in the stream. 
> Accordingly,
> 	they MUST be given at timestamp not later than that 
> given to those
> 	other units. It may be desirable to resend the sample 
> descriptions
> 	periodically, or to use other transport resiliency mechanisms.
> 
> since the text in the draft implies that a copy of the sample 
> description is sent in every packet, rather than allowing it 
> to be sent once and referenced by later packets.
> 

I'll go on with this when the above is cleared out.

Sorry for the long email, all this is something we haven't thought of before, a left over from when sample desc didn't get a timestamp assigned, but where kept in the buffer until they were "inactivated" by newer sample desc.

I would appreciate some feedback

José

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: Dave Singer <[email-masked]>
Date: Thu, 14 Apr 2005 11:23:17 -0700

At 6:22 PM +0200 4/14/05, Jose Rey wrote:
>While understanding the consequences of this I 
>have found a different issue.  Before going into 
>it this is my understanding how reception should 
>work: packets arrive at RTP client, units are 
>extracted from payloads and assigned timestamp, 
>also sample descriptions.  Then they are ordered 
>in the buffer according to TS.  The server 
>provides that the units have the required sample 
>descriptions available at the time of decoding 
>by assigning same timestamp.  Then we'll end up 
>with a situation where the same sample 
>description is in the buffer several times with 
>different timestamp values, is this OK?

yes, they have the same SIDX and the same 
content, so they replace each other or not as you 
like to think;  it's immaterial.

>
>The question is how the (real-time) decoder 
>buffers SDs.  I don't know how this actually 
>happens.  However, regardless of whether we 
>change the timestamp of the SD to that of the 
>RTP packet they are in, as proposed in my 
>previous email or keep it as is, we have to 
>REQUIRE a minimum buffering time for the SDs at 
>the DECODER using dynamic SDs.  I.e., it cannot 
>be that SDs be just discarded once they are used.

I am not sure what you mean by that.  That become 
active until their index goes out of range...

>Because if an SD is present in an aggregate 
>payload applying to several samples, this 
>buffering time is determined by the timestamp of 
>the last text sample in THAT payload that uses 
>that SD; otherwise if the SD applies only to the 
>last text sample, and since the SD is given an 
>earlier timestamp, this is not available anymore 
>as the last sample is to be displayed. 
>
>a) new fields that encode it in the TYPE 5 unit 
>header.  However, we cannot specify a duration 
>for the SD because the timed text format does 
>not understand that.
>b) we REQUIRE the decoder to use the dynamic 
>SIDX algo to know when a SD should be kept or 
>deleted.  Is this possible?  (it is not the RTP 
>client but the decoder that should implement it)

that's what I expected.  SDs are kept until they 
are inaccessible or overwritten.

>c) or we could avoid all this by repeating the 
>SD for every single sample that uses it and so 
>could SDs be discarded after use. However it 
>could potentially be too much overhead, but gain 
>in simplicity.

ugh

>Sorry for the long email, all this is something 
>we haven't thought of before, a left over from 
>when sample desc didn't get a timestamp 
>assigned, but where kept in the buffer until 
>they were "inactivated" by newer sample desc.
>

-- 
David Singer
Apple Computer/QuickTime

---

From: Jan vanderMeer <[email-masked]>
Date: Fri, 15 Apr 2005 12:44:29 +0200

Jose,

> I.e., it cannot be that SDs be just discarded once they are used.

Right, but in my view they are stored in an SIDX buffer until they are 
invalidated; when that is is not relevant. Or do I miss something ?

Best regards,

Jan

"Jose Rey" <[email-masked]>
14-04-2005 18:22
 
        To:     "Colin Perkins" <[email-masked]>
"Dave Singer" <[email-masked]>
Jan vanderMeer/EHV/IPS/PHILIPS@PHILIPS
        cc:     <[email-masked]>
"Magnus Westerlund" <[email-masked]>
        Subject:        RE: [AVT] Update (-09) of Timed Tex draft (WAS: 
RE:Commentsondraft-ietf-avt-rtp-3gpp-timed-text-08.txt)
        Classification: 

[I include Dave and Jan]

Colin,

[snip]

[snip]

The sentence is OK.

While understanding the consequences of this I have found a different 
issue.  Before going into it this is my understanding how reception should 
work: packets arrive at RTP client, units are extracted from payloads and 
assigned timestamp, also sample descriptions.  Then they are ordered in 
the buffer according to TS.  The server provides that the units have the 
required sample descriptions available at the time of decoding by 
assigning same timestamp.  Then we'll end up with a situation where the 
same sample description is in the buffer several times with different 
timestamp values, is this OK? 

The question is how the (real-time) decoder buffers SDs.  I don't know how 
this actually happens.  However, regardless of whether we change the 
timestamp of the SD to that of the RTP packet they are in, as proposed in 
my previous email or keep it as is, we have to REQUIRE a minimum buffering 
time for the SDs at the DECODER using dynamic SDs.  I.e., it cannot be 
that SDs be just discarded once they are used.  Because if an SD is 
present in an aggregate payload applying to several samples, this 
buffering time is determined by the timestamp of the last text sample in 
THAT payload that uses that SD; otherwise if the SD applies only to the 
last text sample, and since the SD is given an earlier timestamp, this is 
not available anymore as the last sample is to be displayed. 

For specifying the buffering time of dynamic SDs we have following 
possibilities:
a) new fields that encode it in the TYPE 5 unit header.  However, we 
cannot specify a duration for the SD because the timed text format does 
not understand that.
b) we REQUIRE the decoder to use the dynamic SIDX algo to know when a SD 
should be kept or deleted.  Is this possible?  (it is not the RTP client 
but the decoder that should implement it)
c) or we could avoid all this by repeating the SD for every single sample 
that uses it and so could SDs be discarded after use. However it could 
potentially be too much overhead, but gain in simplicity.

David, Jan, comments?

Sorry for the long email, all this is something we haven't thought of 
before, a left over from when sample desc didn't get a timestamp assigned, 
but where kept in the buffer until they were "inactivated" by newer sample 
desc.

José