Subject: [MMUSIC] draft-rfc2326-06 comments wrt live streaming

From: Geetha Srikantan <[email-masked]>
Date: Fri, 19 Mar 2004 09:57:15 -0800

This draft addresses the on-demand streaming case very well, however
I believe there are some issues in live streaming that need clarification.
The issues are in the use of the RTP-Info header, the Range header and
ssrc parameter in the Transport header. The points are related, so I 
will cover
them in a single message.

Section 11.3, page 44: 4th paragragh:
'...SHOULD also include the RTP-Info header..'

This requirement is fine for on-demand content, however it creates 
complications
for live media. There are many reasons why the RTP-Info cannot be populated
accurately by the server for live content:
(a) Arbitrary delays between live source and server - If there are
    several relay nodes between source and server, there maybe a skew
    in system clocks of each node that tries to correlate
    NPT <-> RTP timestamp (i.e. Range header <-> RTP-Info's rtptime).
(b) Lost packets between live source and server, imply that the server
    cannot accurately predict the next packet's sequence number.

For these reasons, the RTP-Info header is not very useful for live
streaming, and RTCP-based stream synchronization is preferred for live
streams. In any case RTP-Info is only useful for initial stream
synchronization. During a long-running live session the client has to
rely on RTCP for ongoing stream synchronization.
This is a motivation to drop RTP-Info entirely for live streams.

Section 11.4, page 45:
2nd last paragraph:

Range header for live streams:
There is a subtle difference in the semantics of the Range header for live
streams versus on-demand streams. For on-demand streams, the Range
header's start value refers to the presentation time, i.e. NPT of the 
presentation.
For live streams, the Range header's start value indicates either
(a) the duration of time between the start of the live session and the
    time when this Range header was created, or,
(b) the wall-clock time at the server responding to PLAY requests.

Also the main purpose of the Range header is to map the NPT to
the RTP timestamp. As discussed above this mapping is not terribly
accurate in several live streaming scenarios.

If the live streams are part of a multicast session then there would
be an indication of the time when the session is active in the SDP,
however, with clock skews etc, it is diffcult to accurately pinpoint
the instantaneous time in the session.

If the start time of the live session is not known, then 'npt=now-'
is the only reasonable value for the Range header in a PLAYresponse.

For this reason, it would be good to *not* require a Range header
other than 'Range: npt=now-' for live streams.

Perhaps this could also be the mechanism to signal to the client that
this is a live stream and that it needs to perform stream
synchronization strictly via RTCP sender reports.

Section 14.40, page 91:
ssrc:
The transport header in the server's response cannot populate the ssrc for
each stream, accurately in certain scenarios.
The source of the live stream may not be RTSP-aware, and the server has 
to rely
on RTP/RTCP to determine ssrc for each stream.
The server may have joined a multicast live session and receives, say, 
one of the streams.
It also receives SETUP request from a client and needs to respond to 
SETUP from a client,
before it sees the first packet of each stream.
Dropping the ssrc parameter from the Transport header in the SETUP 
response would
simplify this case.

thanks
geetha

_______________________________________________
mmusic mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mmusic

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 23 Mar 2004 17:01:12 +0100

Hi Geetha,

Thanks for the comments. See comments inline.

>     NPT <-> RTP timestamp (i.e. Range header <-> RTP-Info's rtptime).

I don't see how this clock skew effects anything. The binding in a PLAY 
response of RTP-Info and Range header is not dependent on at what time 
you receive the play response or the media stream. The synchronization 
mechanism in both RTCP and in RTSP are not dependent on any 
synchronization of the receiver clock to any other clock. One binds all 
present media streams to a common clock reference. In most case for RTSP 
the NPT time. In RTSP this is accomplished by giving the corresponding 
RTP TS for a given NPT time.

In fact any media streams properly timestamped can be synchronized as 
long as any data point for synchronization against the common time scale 
exist.

I don't see any reason why there would be skew even in a multi-tire 
distribution system. When the first host has established the 
synchronization information he can forward it, without any dependency of 
its local clock.

>     cannot accurately predict the next packet's sequence number.

Also here I can't see the problem. The RTP sequence number indicates the 
first sequence number for a stream that the given synchronization 
information and time range applies for. Thus if one receives a RTP 
packet with a higher sequence number one knows that it is for the new 
range.

A relay server that has received at least one packet can give this 
information that is valid even if a several RTP packets is lost before 
one gets forward. The client can't determine if the loss is between the 
relay or before.

And if the server really has not received either the information from a 
upstream source or any RTP packet when responding, this value can be 
excluded. The RTP-Info header only requires either of the values to be 
present.

Also if one really can't establish the information one is allowed to 
excluded. It is after all a SHOULD not a MUST. However in general it 
seems reasonable that a server will be able to provide this information.

> This is a motivation to drop RTP-Info entirely for live streams.

What is the problem with long run session? If the original data source 
does not have a problem with its clock their should not be any clock 
skew entered into the RTP timestamp. And if that occur this can be 
corrected by RTCP sender reports. Any local clock skew between media 
will need to be detected and dealt by the client locally.

> (b) the wall-clock time at the server responding to PLAY requests.

In both these cases the time is as viewed by the origin server. In a 
relay structure, it is not the local time of each relay. Using RTSP for 
relay will provide all relay servers also with the correct view of the 
NPT or wall clock time of the origin server.

> accurate in several live streaming scenarios.

As I don't think it is any less accurate then for on-demand media I 
don't understand what the problem is.

> the instantaneous time in the session.

The instantaneous time is calculated based of the RTP timestamp for a 
given media and the mapping received by Range and RTP-Info between RTP 
timestamp and NPT at any point

Example:

  Sync point
-----|------------------------------------x------> t
TS   A                                    B

When one started the session one received by Range and RTP-Info the 
following information for RTP TS A.

PLAY response:
Range: npt=56.97-
RTP-Info: url=<snip>; rtptime=164400

Thus A = 1644000
The media timestamp rate is 90kHz.

The NPT time for B=465357 is:

(465357-164400)/90000 + 56.97 = 60,3139666 s

> is the only reasonable value for the Range header in a PLAYresponse.

I agree that if one truly does not know the time since session start it 
is a viable operation. However all functionality exist to know the time.

> other than 'Range: npt=now-' for live streams.

No, I think there are a lot of application that can benefit from 
presenting the original wall clock or time since session start.

> synchronization strictly via RTCP sender reports.

There are already text in there to indicate that the session has live 
qualities. There are already reasons why a client must support 
synchronization by RTCP. For example sender side clock skew corrections, 
multiple sources using different SSRCs, the possibility to exclude 
RTP-Info header. However the RTP-Info+Range header is a reliable 
transferred information. Although not guaranteed to be timely, it has a 
rather high probability to reach the client prior to any RTCP message. 
Also in most case it should reach the client prior to it finishing 
initial buffering, thus providing it with good initial synchronization 
for all media streams.

> 
> Section 14.40, page 91:
> ssrc:
> The transport header in the server's response cannot populate the ssrc
> for each stream, accurately in certain scenarios.
> to rely on RTP/RTCP to determine ssrc for each stream.

It is possible to skip the SSRC, however not recommend as it indicates 
for which SSRC the synchronization information in RTP-Info applies to. 
Thus I would recommend that server awaits at least the first RTP packet 
to determine what SSRC is used for media distribution.

Cheers

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

This communication is confidential and intended solely for the addressee(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you believe this message has been sent to you in error, please notify the sender by replying to this transmission and delete the message without disclosing it. Thank you.

E-mail including attachments is susceptible to data corruption, interruption, unauthorized amendment, tampering and viruses, and we only send and receive e-mails on the basis that we are not liable for any such corruption, interception, amendment, tampering or viruses or any consequences thereof.

---

From: Geetha Srikantan <[email-masked]>
Date: Thu, 25 Mar 2004 11:24:12 -0800

hi Magnus,

I'm still reviewing your response. A couple of questions about your 
response..

Are you assuming :

(a) That all live sources are RTSP-aware?

(b) The server could maintain it's own RTP clock for each stream by 
observing
     the RTP timestamps of packets on each stream?

(c) The live source's transmission schedule for packets follows the RTP 
timestamp clock?
     That is, each succeeding packet's RTP timestamp will be equal to or 
greater than all previous
     packets, for that stream (modulo rollover)?

thanks
geetha

Magnus Westerlund wrote:

> I don't see how this clock skew effects anything. The binding in a 
> PLAY response of RTP-Info and Range header is not dependent on at what 
> time you receive the play response or the media stream. The 
> synchronization mechanism in both RTCP and in RTSP are not dependent 
> on any synchronization of the receiver clock to any other clock. One 
> binds all present media streams to a common clock reference. In most 
> case for RTSP the NPT time. In RTSP this is accomplished by giving the 
> corresponding RTP TS for a given NPT time.
>
> long as any data point for synchronization against the common time 
> scale exist.
>
> synchronization information he can forward it, without any dependency 
> of its local clock.
>
> Also here I can't see the problem. The RTP sequence number indicates 
> the first sequence number for a stream that the given synchronization 
> one gets forward. The client can't determine if the loss is between 
> the relay or before.
>
> And if the server really has not received either the information from 
> a upstream source or any RTP packet when responding, this value can be 
> relay structure, it is not the local time of each relay. Using RTSP 
> for relay will provide all relay servers also with the correct view of 
> the NPT or wall clock time of the origin server.
>
> I agree that if one truly does not know the time since session start 
> it is a viable operation. However all functionality exist to know the 
> time.
>
> synchronization by RTCP. For example sender side clock skew 
> corrections, multiple sources using different SSRCs, the possibility 
> to exclude RTP-Info header. However the RTP-Info+Range header is a 
> reliable transferred information. Although not guaranteed to be 
> timely, it has a rather high probability to reach the client prior to 
> any RTCP message. Also in most case it should reach the client prior 
> to it finishing initial buffering, thus providing it with good initial 
> synchronization for all media streams.
>
>>
>> Section 14.40, page 91:
>> ssrc:
>> The source of the live stream may not be RTSP-aware, and the server 
>> has to rely on RTP/RTCP to determine ssrc for each stream.
>> The server may have joined a multicast live session and receives, 
>> say, one of the streams.
> Thus I would recommend that server awaits at least the first RTP 
> packet to determine what SSRC is used for media distribution.
>
>
> Cheers
>
> Magnus Westerlund
>
> This communication is confidential and intended solely for the 
> addressee(s). Any unauthorized review, use, disclosure or distribution 
> is prohibited. If you believe this message has been sent to you in 
> error, please notify the sender by replying to this transmission and 
> delete the message without disclosing it. Thank you.
>
> E-mail including attachments is susceptible to data corruption, 
> interruption, unauthorized amendment, tampering and viruses, and we 
> only send and receive e-mails on the basis that we are not liable for 
> any such corruption, interception, amendment, tampering or viruses or 
> any consequences thereof.
>
>

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 26 Mar 2004 14:23:02 +0100

Hi Geetha,

Answers below to what I assume.

> (a) That all live sources are RTSP-aware?

No, but that the there is a root-distributor that is a RTSP server. The 
root RTSP server can be synchronized using either standard RTP means or 
any proprietary solution as this interface is not standardized. However 
it will be this RTSP root server that will be responsible giving the 
stream a NPT time, unless that is also provided through the interface 
between RTSP and the live source.

>     the RTP timestamps of packets on each stream?

I don't see how that this would be necessary. If one has a RTSP server 
that receives a RTP + RTCP media stream. It will after listened to this 
for a few seconds received all necessary synchronization information. If 
this is not the root server and is only a relay it can use RTSP to 
receive the synchronization information for the media streams it is 
forwarding directly in RTSP.

>     packets, for that stream (modulo rollover)?

I am only assuming that this is a normal RTP media stream. Thus it can 
have backwards jump and other non-monotonic behaviour in the timestamp 
field. However I do expect that the RTP packets are sent in order, but 
not received by any relay in order. Thus a relay will be required to 
take certain care against reordering in the RTP-Info sequence number field.

I am however relying on that RTCP stream contains SRs for all media 
streams sent to allow the server to make inter media synchronization, 
and to be able to correctly provide the NPT to TS mapping.

Cheers

Magnus Westerlund

---

From: Geetha Srikantan <[email-masked]>
Date: Tue, 30 Mar 2004 18:25:14 -0800

hi Magnus,

Thanks for clarifying your assumptions. This is helpful.
More inline..

Magnus Westerlund wrote:

> No, but that the there is a root-distributor that is a RTSP server. 
> The root RTSP server can be synchronized using either standard RTP 
> means or any proprietary solution as this interface is not 
> standardized. However it will be this RTSP root server that will be 
> responsible giving the stream a NPT time, unless that is also provided 
> through the interface between RTSP and the live source.

I was assuming the same thing.

> that receives a RTP + RTCP media stream. It will after listened to 
> this for a few seconds received all necessary synchronization 
> information. If this is not the root server and is only a relay it can 
> use RTSP to receive the synchronization information for the media 
> streams it is forwarding directly in RTSP.
>
This is an important point of concern.
I agree that an RTSP server, after it has received RTCP SR for each 
stream, can map TS to NPT.
There are some issues though:

1. The server may not see RTCP SRs for all streams in the RTSP session 
for several seconds,
   because of (a) staggered transmission of RTCP SRs of each stream, (b) 
lost SRs.
   Since RTCP transmission intervals can be 5-10 seconds, this could be 
10-20 seconds or so.

Now the server could either
(a) deny all client requests for the live session until it receives RTCP 
SRs for all streams - even
though the server has RTP packets for all streams. This implies that 
clients won't have access
to the live stream for several seconds after the session was requested.
OR
(b) respond to client requests without the RTP-Info header in PLAY 
responses.
 
If (a) and (b) are accepted as likely scenarios, then the clients cannot 
expect to see a RTP-Info header
in PLAY responses, and should be able to handle the absence of the 
RTP-Info header.

2. The server may not know when the live session started, for example if 
it joins an ongoing
    live session.
One option is for the server to assume NPT=0 at the time it receives RTP 
packets from the
live source, and use that in synchronization mappings. So, the Range 
header would have
this server-interpreted version of NPT.

>> (c) The live source's transmission schedule for packets follows the 
>> RTP timestamp clock?
>>     That is, each succeeding packet's RTP timestamp will be equal to 
>> or greater than all previous

Reordering and lost packets make it difficult to populate the sequence 
number field of RTP-Info headers
for live streams. This is not the case for stored content. I think this 
is a point worth noting for live streaming.
If the server has to guess about the next packet it is going to send, it 
can't be a good thing.

thanks
geetha

---

From: Geetha Srikantan <[email-masked]>
Date: Tue, 30 Mar 2004 18:59:21 -0800

Magnus,

I see your point of view on this, since you clarified your assumptions.

In previous offline discussions there were discussions about how a RTSP 
server would
maintain timing info for live streams. My confusion was centered around 
that.

I believe RTCP SR-based synchronization is possible, as you pointed 
out..however it does
have some issues, as mentioned previously.

Replying inline to some of the points in this mesage...

> excluded. The RTP-Info header only requires either of the values to be 

I did not realize that the RTP-Info header only requires either of the 
values to be present.
I don't see it in section 14.33 of the draft..am I missing something?

> seems reasonable that a server will be able to provide this information.

True..

> the NPT or wall clock time of the origin server.

Ok..

> don't understand what the problem is.

I agree that RTCP SR-based mapping of NPT to TS is accurate..except that 
this mapping
is not possible until an SR is received for each stream of the RTSP session.

> (465357-164400)/90000 + 56.97 = 60,3139666 s
This is fine as long as a non-zero and numeric Range header value exists.
In the case where the live source is not RTSP-aware, there is no Range 
header available
and sync point A is effectively NPT=0, assuming the server started 
accessing the live
session at A.

> it is a viable operation. 

You mean that npt=now- is acceptable in the Range header in this case, 
right?

> However all functionality exist to know the time.

Did you mean that the timestamp to NPT mapping is possible as you 
described above?

> presenting the original wall clock or time since session start.

As long as there is an understanding that the session start may not be 
an accurate value, as in cases
where it is not known.

> synchronization for all media streams.

Yes, I agree that RTP-Info+Range can be helpful to the client since they 
are transmitted over RTSP/TCP.
My point is that if there are inaccuracies in either RTP-Info or Range 
because of unknown quantities like
session start time, missing packets, these need to be tolerated by clients.

> packet to determine what SSRC is used for media distribution.

I am not convinced that the user experience is good when a server 
refuses clients access to live sessions until
it has access to all the stream synchronization information. If the 
server were to send RTP and RTCP packets,
without the RTP-Info or SSRC in RTSP responses, the client has an 
opportunity to start decoding each stream
independently and perform inter-stream synchronization as soon as the 
client receivess one RTCP SR per stream.
There is at least some media being rendered at the client, even if it is 
not lip-sync'd etc.

thanks
geetha