Subject: Time alignment draft and RTP packet delay -- Was Re: R:[AVT] CCMdraftreview--should"bandwidth"includetheheaderoverhead

From: "Desineni, Harikishan" <[email-masked]>
Date: Fri, 2 Mar 2007 14:01:24 -0800

Hi Tom,

I just read your time alignment draft available at http://tools.ietf.org/id/draft-taylor-avt-time-align-00.txt

The term "misalignment" does not make much sense in packet switched networks. IMO, capturing delayed or early arrival of RTP packets makes more sense in packet switched networks. The format of RTCP feedback message defined in this draft can be used for this purpose. The magnitude of the receiver reported delayed or early arrival of RTP packets can be used by the sender to perform rate adaptation on its audio/video media streams. How a sender can further act on its stream can be codec-specific as stated in the draft.

comments?

> -----Original Message-----
> From: Tom-PT Taylor [mailto:[email-masked]]
> Sent: Friday, February 23, 2007 10:52 AM
> To: Desineni, Harikishan
> Cc: Even, Roni; Allison, Art; Magnus Westerlund; [email-masked]
> Subject: Re: R:[AVT] CCMdraftreview--
> should"bandwidth"includetheheaderoverhead
> 
> Maybe the time adjustment draft, draft-taylor-avt-time-align-00.txt,
> would be an useful mechanism here. I plan to reissue it shortly.
> 
> Desineni, Harikishan wrote:
> > I am addressing Art’s point on a point to point topology. A sender
> > can react to TMMBR several ways as you pointed (changing frame rate,
> > packet sizes etc,). The end result could be that packets are getting
> > delayed or arriving too early at the receiver. This could happen due
> > to variable size of video packets. Some video packets may experience
> > longer delay than the corresponding audio packets or audio packets
> > may experience more delay when they stay behind video over serial
> > links (Guido’s draft).
> >
> >
> >
> > My point is that a receiver could inform the sender about delayed or
> > too early arrival of packets if it chooses to. This can be done with
> > the help of a new RTCP message. A sender can (if it chooses to) make
> > use of the delay/earliness information while adjusting its rate for
> > each media.
> >
> >
> >
> > Regarding the feedback loop issue:
> >
> > What I am considering is another RTCP message for reporting delay
> > when a receiver detects problem with packet arrivals. The performance
> > of feedback is anyways dictated by the RTCP timing rules allowed by
> > the RTP profile under use. This is applicable to any RTCP message.
> >
> >
> >
> > Thanks,
> >
> > Hari
> >
> >
> >
> > ________________________________
> >
> > From: Even, Roni [mailto:[email-masked]] Sent: Friday,
> > February 23, 2007 7:22 AM To: Allison, Art; Magnus Westerlund;
> > Desineni, Harikishan Cc: [email-masked] Subject: RE: R:[AVT]
> > CCMdraftreview-- should"bandwidth"includetheheaderoverhead
> >
> >
> >
> > Hi,
> >
> > I think what Magnus is saying is that TMMBR address the bit rate of a
> > specific stream, for example the video.
> >
> > Now as for lip-synch problem. The assumption is that the sender will
> > put the audio and video in its output buffer synchronizes enableing
> > the receiver to adjuct based on time. The sender do not have any
> > control on the way the medai handled by the network.
> >
> > When a receiver aks for a TMMBR to reduce the bit rate, in some cases
> > due to application constrains it is typicaly for the video. The
> > sender can reduce the bit rate and keep the frame rate and this will
> > keep the lip synch or it may reduce the frame rate and re-adjuct is
> > internal synchronization between the audio and video keeping them
> > synchronized on its output buffers.
> >
> > So my view, for TMMBR, lip-synch is not related
> >
> > Roni Even
> >
> >
> >
> > ________________________________
> >
> > מאת: Allison, Art [mailto:[email-masked]] נשלח: ו 23/02/2007 16:54
> > אל: Magnus Westerlund; Desineni, Harikishan עותק לידיעה: [email-masked]
> >  נושא: RE: R:[AVT] CCMdraftreview--
> > should"bandwidth"includetheheaderoverhead
> >
> > How to control lip-sync or lack thereof is clearly out of scope here.
> >
> >
> > However, I respectively disagree with the AD that it is out of scope
> > as a factor in the decision between selection of a total bit rate
> > control vs. a partial bit rate control.
> >
> > It is only one factor, but declaring it out of scope as a
> > consideration can be expected to impact the technical choice.
> >
> > I request reconsideration and clarification, as surly the AD did not
> > intend to force the group to ignore a technical factor that is
> > material to the RFC content.
> >
> > _____________ Art Allison Director, Advanced Engineering Science &
> > Technology National Association of Broadcasters 1771 N Street, NW
> > Washington, D.C. 20036 Phone: 202.429.5418 Fax: 202.777.4981
> > [email-masked]
> >
> > The National Association of Broadcasters is a trade association that
> > advocates on behalf of more than 8,300 free, local radio and
> > television stations and also broadcast networks before Congress, the
> > Federal Communications Commission and the Courts.
> >
> > -----Original Message----- From: Magnus Westerlund
> > [mailto:[email-masked]] Sent: Friday, February 23,
> > 2007 4:12 AM To: Desineni, Harikishan Cc: Allison, Art; [email-masked]
> > Subject: Re: R: [AVT] CCMdraftreview--
> > should"bandwidth"includetheheaderoverhead
> >
> > Hi,
> >
> > I don't think this topic is really relevant for the TMMBR discussion.
> >  The issue you talk about arise independently of the existance of
> > TMMBR or not. Beyond how a media sender packetize data it can't make
> > much to affect the lip-synch (assuming a good implementation). RTP
> > was designed with a model that it is the receiver that needs to
> > buffer and delay its playout to ensure best possible synchronization
> > and smooth playout. And as I see it there are good reasons for this
> > model. First any sender adjusting model breaks down in multi-party
> > sessions. Also the feedback loop issues are eliminated by having the
> > receiver deal with it.
> >
> > Cheers
> >
> > Magnus Westerlund
> >
> > IETF Transport Area Director & TSVWG Chair
> > ----------------------------------------------------------------------
> >  Multimedia Technologies, Ericsson Research EAB/TVA/A
> > ----------------------------------------------------------------------
> >  Ericsson AB                | Phone +46 8 4048287 Torshamsgatan 23
> > | Fax   +46 8 7575550 S-164 80 Stockholm, Sweden | mailto:
> > [email-masked]
> > ----------------------------------------------------------------------
> >
> >
> > _______________________________________________ Audio/Video Transport
> > Working Group [email-masked] https://www1.ietf.org/mailman/listinfo/avt
> >
> >
> >

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: "Allison, Art" <[email-masked]>
Date: Fri, 9 Mar 2007 10:36:26 -0500

Having a specific tool to enable direct feedback when the receiver detects differential delay such as the RTCP message suggested by Hari below seems to be a good and direct approach. 

The time stamp adjustment process in http://tools.ietf.org/id/draft-taylor-avt-time-align-00.txt does not seem optimal for a source feeding many sinks. As the draft points out, when time stamps are adjusted content is usually lost. Restamping may be very effective for real-time encoding control (and 1 to 1 connections), but when the server has already encoded content, that is feeding many unicasts, rate control seems much less complex to implement.  

When a connection is established, does the capacity remain constant for hundreds of milliseconds  to seconds? 
If the differential variability is faster than this, ensuring the buffer models can accommodate the changes would seem better, but buffers need to get larger and larger as the variances grow. Lower buffer sizes would be needed with a direct the rate control signaling to directly address time to time change in delivery path delays when they exist. Of course the number of bit-rate choices available to a server can also impact the ability to find a rate that meets the buffer model, that is enable a the audio and video to arrive by or not too far before presentation time arrives. (I assume the zero-decide time buffer model as per MPEG systems modeling and that there are several encoded versions of audio and video from which the server chooses based on pipe size and highest quality consistent with that pipe size is an objective.) 

To be clear, I continue to post to try to alert this community to the implementation reality that lip synch requires lots of effort to control.  With constant bitrate exactly set,  a one-way defined path, and all experts asserting it 'should' work; lip sync should be easy. It has not been, even in a one-hop distribution system....due to many factors (pointed documents with reasons previously). 
 
Thanks for your time,
Art
_____________
Art Allison
Director, Advanced Engineering
Science & Technology
National Association of Broadcasters
1771 N Street, NW
Washington, D.C. 20036
Phone: 202.429.5418
Fax: 202.777.4981
[email-masked]

The National Association of Broadcasters is a trade association that advocates on behalf of more than 8,300 free, local radio and television stations and also broadcast networks before Congress, the Federal Communications Commission and the Courts.

-----Original Message-----
From: Desineni, Harikishan [mailto:[email-masked]] 
Sent: Friday, March 02, 2007 5:01 PM
To: Tom-PT Taylor
Cc: Even, Roni; Allison, Art; Magnus Westerlund; [email-masked]
Subject: Time alignment draft and RTP packet delay -- Was Re: R:[AVT] CCMdraftreview--should"bandwidth"includetheheaderoverhead

Hi Tom,

The term "misalignment" does not make much sense in packet switched networks. IMO, capturing delayed or early arrival of RTP packets makes more sense in packet switched networks. 

The format of RTCP feedback message defined in this draft can be used for this purpose. The magnitude of the receiver reported delayed or early arrival of RTP packets can be used by the sender to perform rate adaptation on its audio/video media streams. How a sender can further act on its stream can be codec-specific as stated in the draft.

comments?

> > when a receiver detects problem with packet arrivals. The 
> > performance of feedback is anyways dictated by the RTCP timing rules 
> > allowed by the RTP profile under use. This is applicable to any RTCP message.
> >
> >
> >
> > Thanks,
> >
> > Hari
> >
> >
> >
> > I think what Magnus is saying is that TMMBR address the bit rate of 
> > a specific stream, for example the video.
> >
> > When a receiver aks for a TMMBR to reduce the bit rate, in some 
> > cases due to application constrains it is typicaly for the video. 
> > The sender can reduce the bit rate and keep the frame rate and this 
> > will keep the lip synch or it may reduce the frame rate and 
> > re-adjuct is internal synchronization between the audio and video 
> > keeping them synchronized on its output buffers.
> >
> > --------------------------------------------------------------------
> > --  Multimedia Technologies, Ericsson Research EAB/TVA/A
> > _______________________________________________ Audio/Video 
> > Transport Working Group [email-masked]