Subject: RE: [AVT] H.26L packetization: the two properties of the RTP time stamp

From: "Even, Roni" <[email-masked]>
Date: Mon, 11 Mar 2002 11:16:07 +0200

Stephan,
The timestamp is used not only for jitter but also for lip-synch in video
conferencing. Another need for the timestamp as defined in RTP spec is to
give the right timing when going from the packet network to the switched
network like GW from H.323 to H.320. The need is for the RTP like timestamp
and not the RFC2250 definition. Having the timestamp in the stream like
MPEG1/2 will complicate gateway design, thus we prefer to have the
decode/presentation timestamp in the header

Regards
Roni Even
Polycom

> -----Original Message-----
> From: Stephan Wenger [mailto:[email-masked]]
> Sent: Thursday, March 07, 2002 6:11 PM
> To: [email-masked]
> Cc: [email-masked]; thomas Stockhammer
> Subject: [AVT] H.26L packetization: the two properties of the 
> RTP timestamp
> 
> 
> Folks,
> 
> I have started this new thread in order to discuss one observation
> we have made when writing the H.26L packetization draft.  The
> issue, however, may also apply to other forthcoming packetization
> schemes.
> 
> The RTP timestamp seems to have two properties (and I almost
> hope that someone corrects me and says that one of the two is
> a mis-interpretation of RFC1889):
> a) To convey media timing information (often sampling instant,
>    sometimes presentation time, both are equivalent wrt. the
>    following discussions)
> b) To implement the jitter buffer management.  Citing RFC1889:
>    Section 6.3.1 (RTCP), paragraph on arrival jitter : 
> "[...]the relative
>    transit time is the difference between a packet's RTP timestamp
>    and the receiver's clock at the time of arrival, measured 
> in the same
>    units.
> 
> For continuous media streams, say G.711 audio, such a coupling
> of functionalities is not problematic.  As thousands of 
> implementations
> show us, it works very well.
> 
> As soon as features such as B-frames come into the picture (where
> the timestamp does not increase monotically with the sequence
> number), an RTP payload spec designer has two choices: stick
> to the "sampling instant" concept for the timestamp and de facto
> require much larger jitter buffers (by allowing higher 
> jitter), or bind
> the timestamp to something more appropriate than the sampling
> time.  The former seems to make media synchronization easier
> but increases delay, the latter may reduce delay but disallows for
> perfect synchronization.
> 
> When designing RFC 2429 (H.263+ payload spec), for example,
> we decided to use the "presentation time" because of the sync-
> problems associated with layered coding, and because no one
> with a sane mind would use B-frames in a low-delay environment
> anyway (hence the additional delay of a big jitter buffer seemed
> acceptable).
> 
> H.26L is the first simple (non multiplexed in the sense of MPEG)
> media codec I'm aware of that has the capability of de-coupling
> the transmission time from the sampling/presentation time (though
> not decoding time).  For some packets, neither the sampling nor
> the presentation time in the very sense of the words are actually
> defined.  In the example in the draft, the packets of the commercial
> were sampled at an unknown time, and are presented at an unknown
> time (probably never) but they are transmitted in the stream.
> 
> When we wrote the draft, we looked at other payload RFCs and found
> the solution of RFC2250 which perfectly solves the jitter buffer
> and jitter calculation problem. Hence, we used it.  However, 
> that means
> that we have to rely for exact timing information to the media stream
> itself ONLY -- though the RTP timestamp can be used to roughly
> synchronize media streams that do not use Enhanced RPS.  Same is
> IMHO the case for RFC2250.
> 
> Currently, I do not see a better solution.
> 
> Stephan
> 
> 
> _______________________________________________
> Audio/Video Transport Working Group
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/avt
> 

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: Stephan Wenger <[email-masked]>
Date: Mon, 11 Mar 2002 14:19:01 +0100

Hi Roni,

At 11:16 AM 3/11/2002 +0200, you wrote:
>conferencing.

I'm aware of that (current) property.  And I'm very much in favor of
having a similar property for such applications which need it.

>Having the timestamp in the stream like MPEG1/2 will complicate
>gateway design, thus we prefer to have the decode/presentation
>timestamp in the header

I see the point, and I have several ideas how the presentation
timestamp could be easily parsable AND allow the RTP timestamp
to be consistent with the buffer model (I believe I made clear why
it cannot be consistent with the buffer model AND with the
presentation time at the same time).

Let's discuss this over lunch in Minneapolis.

Stephan

---

From: "Even, Roni" <[email-masked]>
Date: Tue, 12 Mar 2002 13:36:44 +0200

Hi,
I already sent my comment and included lip-synch as important use of
timestamp.
Thanks
Roni Even
Polycom.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Tuesday, March 12, 2002 12:51 PM
To: Thomas Stockhammer
Cc: [email-masked]; [email-masked]; M. Reha Civanlar;
[email-masked]; [email-masked]; Stephan Wenger
Subject: Re: [AVT] H.26L packetization: the two properties of the RTP
timestamp

Dear all, 

I want to check something before we go on. 

I am concerned that being deeply involved in video -which is more than
understandable ;-) 
we may  forget the *key* feature that RTP should give us: 
the ability to synchronize with other data streams, and especially audio. 

specifically a termimal will receive audio and one audio frame has a time
stamp, 
transported by the RTP TS. 

if it receives video the purpose of the RTP TS is to 
*assign a time stamp to an image* so that an audio/video 
terminal can synchronize audio and video playback. 

do we agree that this is the primary purpose here ? 

regards,

Philippe Gentric
Software Architect
Philips MP4Net
[email-masked]
http://www.mpeg-4.philips.com 

	Thomas Stockhammer <[email-masked]> 
Sent by: [email-masked] 

03/11/2002 14:49 

        To:        Stephan Wenger <[email-masked]> 
        cc:        Jan vanderMeer/EHV/BE/PHILIPS@EMEA3
"M. Reha Civanlar" <[email-masked]>
[email-masked]
[email-masked]
(bcc: Philippe Gentric/LIM/CE/PHILIPS) 
        Subject:        Re: [AVT] H.26L packetization: the two properties of
the RTP timestamp 

        Classification:         

Dear all,

the timestamp seems to be a real animal! Stephan, what do you think :-)?

> The authors of the JVT/H.26L packetization draft thought about
> using the decoding timestamp, but we dismissed it in favor of
> the sending time.  I don't recall the precise reason, but I believe
> we had some uneasiness with the term itself, as it is (so far) not
> defined in JVT.
>
> I would be willing to change the draft to "decoding timestamp", and
> this will solve the dual-property problem of the RTP timestamp --
> assuming that we mean the timestamp for the decoding of one
> slice, not picture.  For the frequently discussed reasons we don't know
> the decoding timestamp of the whole picture until the final slcie
> is sent.
>
> Whether this approach will facilitate media synchronization is another
> problem.  I don't believe it does.  But I'm willing to listen.

I was actually in favor of using the the RTP as DTS when drafting this
proposal.
But we were not sure if this does not cause any other problems. So we went
with

the one which seemed to be less ambigious, RFC2250. However, I still support
very much the RTP = DTS concept unless anyone tells me a real problem.

About buffers and timestamp, I have a kind of naive idea.

What is this all necessary for. The encoder has to produce something
the decoder is capable to decode under some buffer constraints.

The DTS is the time the slice (!, not frame) is removed from the receiver
buffer,
decoded and the decoded slice (!) is moved to the reference frame buffer!
The video has to be coded such that the receiver buffer can take the sum of
all
bits
assigned to all slices with DTS>current_time.

In addition, we have a time-stamp which allows to remove a frame from the
reference frame buffer. This is neither PTS nor DTS but something inherent
in H.26L. The encoder should also take care that the decoder can take
all slices in the reference frame buffer (pixel based).

The PTS is again different. A third buffer might be a display buffer which
takes all frames which are no more in the reference frame buffer but where
not display yet because of the PTS. Is this necessary?

The profiles give upper limits on receiver buffer and reference frame
buffer.

Any problems with this concept?

Best regards

Tom