Subject: Re: [MMUSIC] draft-rfc2326-06 comments wrt live streaming

From: Magnus Westerlund <[email-masked]>
Date: Thu, 01 Apr 2004 11:44:17 +0200

Hi Geetha,

See inline

Geetha Srikantan wrote:

> Magnus,
> 
> I see your point of view on this, since you clarified your assumptions.
> 
> In previous offline discussions there were discussions about how a RTSP 
> server would
> maintain timing info for live streams. My confusion was centered around 
> that.
> 
> I believe RTCP SR-based synchronization is possible, as you pointed 
> out..however it does
> have some issues, as mentioned previously.
> 
> Replying inline to some of the points in this mesage...
> 
> 
> 
> Magnus Westerlund wrote:
> 
>>> (b) Lost packets between live source and server, imply that the server
>>>     cannot accurately predict the next packet's sequence number.
>>
>>
>>
>> Also here I can't see the problem. The RTP sequence number indicates 
>> the first sequence number for a stream that the given synchronization 
>> information and time range applies for. Thus if one receives a RTP 
>> packet with a higher sequence number one knows that it is for the new 
>> range.
>>
>> A relay server that has received at least one packet can give this 
>> information that is valid even if a several RTP packets is lost before 
>> one gets forward. The client can't determine if the loss is between 
>> the relay or before.
>>
>> And if the server really has not received either the information from 
>> a upstream source or any RTP packet when responding, this value can be 
>> excluded. The RTP-Info header only requires either of the values to be 
>> present.
> 
> 
> I did not realize that the RTP-Info header only requires either of the 
> values to be present.
> I don't see it in section 14.33 of the draft..am I missing something?
> 

Please review the ABNF, which specifies that at least one of the must be 
present.

[snip]

>>
>>> Also the main purpose of the Range header is to map the NPT to
>>> the RTP timestamp. As discussed above this mapping is not terribly
>>> accurate in several live streaming scenarios.
>>
>>
>>
>> As I don't think it is any less accurate then for on-demand media I 
>> don't understand what the problem is.
> 
> 
> I agree that RTCP SR-based mapping of NPT to TS is accurate..except that 
> this mapping
> is not possible until an SR is received for each stream of the RTSP 
> session.
> 

Sure, which depending on RTP profile, assigned RTCP SR bandwidth make 
vary significant from usage to usage.

>>> If the live streams are part of a multicast session then there would
>>> be an indication of the time when the session is active in the SDP,
>>> however, with clock skews etc, it is diffcult to accurately pinpoint
>>> the instantaneous time in the session.
>>
>>
>>
>> The instantaneous time is calculated based of the RTP timestamp for a 
>> given media and the mapping received by Range and RTP-Info between RTP 
>> timestamp and NPT at any point
>>
>> Example:
>>
>>  Sync point
>> -----|------------------------------------x------> t
>> TS   A                                    B
>>
>>
>> When one started the session one received by Range and RTP-Info the 
>> following information for RTP TS A.
>>
>> PLAY response:
>> Range: npt=56.97-
>> RTP-Info: url=<snip>; rtptime=164400
>>
>> Thus A = 1644000
>> The media timestamp rate is 90kHz.
>>
>> The NPT time for B=465357 is:
>>
>> (465357-164400)/90000 + 56.97 = 60,3139666 s
>>
> This is fine as long as a non-zero and numeric Range header value exists.
> In the case where the live source is not RTSP-aware, there is no Range 
> header available
> and sync point A is effectively NPT=0, assuming the server started 
> accessing the live
> session at A.

Sure, in the above case the server can set its internal calculation 
mechanism at point A to 0. Then it would probably be best to give a 
npt=now- answer as it is not in the loop to the better timing.

However even if a RTSP play has non numeric, i.e. npt=now- one can 
calculate the NPT from that point and forward. One can even use wall 
clock time and start counting it in relation to the start point.

> 
>>> If the start time of the live session is not known, then 'npt=now-'
>>> is the only reasonable value for the Range header in a PLAYresponse.
>>>
>>
>> I agree that if one truly does not know the time since session start 
>> it is a viable operation. 
> 
> 
> You mean that npt=now- is acceptable in the Range header in this case, 
> right?
> 

Yes.

>> However all functionality exist to know the time.
> 
> 
> Did you mean that the timestamp to NPT mapping is possible as you 
> described above?

What I mean is that one can build solutions that do not have any of 
these problems as long as one uses the right mechanism.

> 
>>> For this reason, it would be good to *not* require a Range header
>>> other than 'Range: npt=now-' for live streams.
>>>
>>
>> No, I think there are a lot of application that can benefit from 
>> presenting the original wall clock or time since session start.
> 
> 
> As long as there is an understanding that the session start may not be 
> an accurate value, as in cases
> where it is not known.

I think that a server should present an accurate time or use npt=now-.

> 
>>> Perhaps this could also be the mechanism to signal to the client that
>>> this is a live stream and that it needs to perform stream
>>> synchronization strictly via RTCP sender reports.
>>>
>>
>> There are already text in there to indicate that the session has live 
>> qualities. There are already reasons why a client must support 
>> synchronization by RTCP. For example sender side clock skew 
>> corrections, multiple sources using different SSRCs, the possibility 
>> to exclude RTP-Info header. However the RTP-Info+Range header is a 
>> reliable transferred information. Although not guaranteed to be 
>> timely, it has a rather high probability to reach the client prior to 
>> any RTCP message. Also in most case it should reach the client prior 
>> to it finishing initial buffering, thus providing it with good initial 
>> synchronization for all media streams.
> 
> 
> Yes, I agree that RTP-Info+Range can be helpful to the client since they 
> are transmitted over RTSP/TCP.
> My point is that if there are inaccuracies in either RTP-Info or Range 
> because of unknown quantities like
> session start time, missing packets, these need to be tolerated by clients.
> 

I don't see how missing packets effects this at all. Session start time 
is either correctly forwarded by, live source, or first hand RTSP 
server, or it will not be correctly given. In which case I would 
recommend that one uses the npt=now- to indicate this.

>>> Section 14.40, page 91:
>>> ssrc:
>>> The transport header in the server's response cannot populate the ssrc
>>> for each stream, accurately in certain scenarios.
>>> The source of the live stream may not be RTSP-aware, and the server 
>>> has to rely on RTP/RTCP to determine ssrc for each stream.
>>> The server may have joined a multicast live session and receives, 
>>> say, one of the streams.
>>> It also receives SETUP request from a client and needs to respond to 
>>> SETUP from a client,
>>> before it sees the first packet of each stream.
>>> Dropping the ssrc parameter from the Transport header in the SETUP 
>>> response would
>>> simplify this case.
>>
>>
>>
>> It is possible to skip the SSRC, however not recommend as it indicates 
>> for which SSRC the synchronization information in RTP-Info applies to. 
>> Thus I would recommend that server awaits at least the first RTP 
>> packet to determine what SSRC is used for media distribution.
> 
> 
> I am not convinced that the user experience is good when a server 
> refuses clients access to live sessions until
> it has access to all the stream synchronization information. If the 
> server were to send RTP and RTCP packets,
> without the RTP-Info or SSRC in RTSP responses, the client has an 
> opportunity to start decoding each stream
> independently and perform inter-stream synchronization as soon as the 
> client receivess one RTCP SR per stream.
> There is at least some media being rendered at the client, even if it is 
> not lip-sync'd etc.
> 

As I wrote in the other mail I don't think this will occur frequently. 
In a well designed system using RTSP the synch information will be 
distributed using TCP, and will basically arrive as soon as possible. If 
there are problems delivering the RTSP response.

A server waiting for the synchronization information can also use a 1xx 
code to tell that it is in fact waiting for the sync info.

I would like to get some feedback from client implementors, it they 
really are willing start without synch and then try to do graceful fix 
of the synchronization when the information arrives. Also without giving 
the RTP-Info TS + Range it will be impossible to present accurate 
session position information. A client will only be able to calculate 
time since media delivery started.

I think a lot of these discussion concerns issues that should be 
discussed in your streaming relay draft to achieve good performance and 
avoid these issues you are so concerned with. From a client perspective 
, I haven't seen a lot of need for clarifications. However I can look on 
how to further improve the discussion in the RTP appendix on synch. 
Otherwise I would recommend if you have specific concerns to present 
proposals for new text.

Cheers

Magnus

-- 

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

_______________________________________________
mmusic mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mmusic

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 01 Apr 2004 11:11:10 +0200

Hi Geetha,

See inline.

Geetha Srikantan wrote:
> Magnus Westerlund wrote:
>>> (a) That all live sources are RTSP-aware?
>>
>>
>>
>> No, but that the there is a root-distributor that is a RTSP server. 
>> The root RTSP server can be synchronized using either standard RTP 
>> means or any proprietary solution as this interface is not 
>> standardized. However it will be this RTSP root server that will be 
>> responsible giving the stream a NPT time, unless that is also provided 
>> through the interface between RTSP and the live source.
> 
> 
> I was assuming the same thing.
> 
>>>
>>> (b) The server could maintain it's own RTP clock for each stream by 
>>> observing
>>>     the RTP timestamps of packets on each stream?
>>>
>>
>> I don't see how that this would be necessary. If one has a RTSP server 
>> that receives a RTP + RTCP media stream. It will after listened to 
>> this for a few seconds received all necessary synchronization 
>> information. If this is not the root server and is only a relay it can 
>> use RTSP to receive the synchronization information for the media 
>> streams it is forwarding directly in RTSP.
>>
> This is an important point of concern.
> I agree that an RTSP server, after it has received RTCP SR for each 
> stream, can map TS to NPT.
> There are some issues though:
> 
> 1. The server may not see RTCP SRs for all streams in the RTSP session 
> for several seconds,
>   because of (a) staggered transmission of RTCP SRs of each stream, (b) 
> lost SRs.
>   Since RTCP transmission intervals can be 5-10 seconds, this could be 
> 10-20 seconds or so.
> 
> Now the server could either
> (a) deny all client requests for the live session until it receives RTCP 
> SRs for all streams - even
> though the server has RTP packets for all streams. This implies that 
> clients won't have access
> to the live stream for several seconds after the session was requested.
> OR
> (b) respond to client requests without the RTP-Info header in PLAY 
> responses.
> 
> If (a) and (b) are accepted as likely scenarios, then the clients cannot 
> expect to see a RTP-Info header
> in PLAY responses, and should be able to handle the absence of the 
> RTP-Info header.

However I don't see it likely that a relay RTSP server would need to 
rely on RTCP to establish the NPT binding. A relay will use the live 
sessions root RTSP sever or another RTSP relay. As it is using RTSP it 
can receive the RTP-Info and Range header which gives it all necessary 
information, unless things are sent using multiple SSRC, in which case 
RTP-Info at the delivery server still can't be used for synch. If the 
relay is slow on responding, I would still recommend that one wait for 
the synch info. Delivering things without synch does not make people 
happy. In conclusion I don't see a good reason why the delivery server 
would not have access to the synchronization information.

> 
> 2. The server may not know when the live session started, for example if 
> it joins an ongoing
>    live session.
> One option is for the server to assume NPT=0 at the time it receives RTP 
> packets from the
> live source, and use that in synchronization mappings. So, the Range 
> header would have
> this server-interpreted version of NPT.
> 

But if the delivery server uses RTSP to request the media, it will 
receive this information.

I would also like to point out that any delay of getting the synch 
information will only be present for the first user of a certain stream. 
Any user requesting a stream already delivered to the delivery server 
will be able to get correct information directly.

>>> (c) The live source's transmission schedule for packets follows the 
>>> RTP timestamp clock?
>>>     That is, each succeeding packet's RTP timestamp will be equal to 
>>> or greater than all previous
>>>     packets, for that stream (modulo rollover)?
>>>
>>
>> I am only assuming that this is a normal RTP media stream. Thus it can 
>> have backwards jump and other non-monotonic behaviour in the timestamp 
>> field. However I do expect that the RTP packets are sent in order, but 
>> not received by any relay in order. Thus a relay will be required to 
>> take certain care against reordering in the RTP-Info sequence number 
>> field.
> 
> 
> Reordering and lost packets make it difficult to populate the sequence 
> number field of RTP-Info headers
> for live streams. This is not the case for stored content. I think this 
> is a point worth noting for live streaming.
> If the server has to guess about the next packet it is going to send, it 
> can't be a good thing.
> 

I don't see how reordering creates any real problem. When a new user 
joins one takes the last sequence number received. Then in the RTP-Info 
one includes this number plus one. Then one forwards the next packet, it 
this was reordered an is in fact an older packet than what one claims in 
the RTP-Info, the client is able to detect that this packet is prior to 
the start of the stream. If there is packet loss or reorder that results 
in that the first packet that arrives to the client has sequence number 
much higher than the indicated one, it is okay. Because the only thing 
the RTP sequence number in the RTP-Info indicates is from what sequence 
number the request media stream is delivered.

Cheers

Magnus Westerlund

---

From: Andrew Large <[email-masked]>
Date: Thu, 01 Apr 2004 10:17:45 -0800

Please forgive me for jumping into the middle of this, but this area is
of considerable concern for us and I feel compelled to offer my $.02...

First, Magnus, do I correctly capture your assertions?

	1. The RTP-Info header need have only the "seq" or "rtptime"
clauses, but not both.

	2. For live streams, the server can provide "rtptime" values
that are meaningful to a client, even in the absence of a meaningful
Range header (e.g., when Range is "npt=now-").

	3. The server should always have access to stream
synchronization information, so having the raw data necessary for
generating the "rtptime" values shouldn't be an issue.

If I have correctly captured your position above, I have a few
questions/comments:

	a. [I apologize for not being on the list early enough to read
the entire dialog] Are you conceding the point that the server may not
know the sequence numbers of the first packets in each stream?

	b. When one populates the "seq" clause, how "close" does it have
to be to the actual value?  If it is acceptable to be off by some
amount, could you provide some concrete parameters?  Either by time or
by sequence number value?  (e.g., packet with given "seq" value must
have RTP timestamp within 500ms of that of the first packet actually
sent in that stream after the PLAY reply; or the first packet's sequence
number must differ by no more than N).

	c. When one populates the "Range" header with a real value, what
constraints are there (if any) on the relationship between that value
and the first packets actually sent in each stream?  For example,
several servers simply report the value requested by the client and then
start their streams with packets whose timestamps fall <= the value in
the range header.  In this case, the "rtptime" clause values in the
RTP-Info header (which must map to the Range header) will be >= the RTP
timestamps of the first packets actually received by the client.  Note
that this behavior is preferable to support (among other things), very
long duration media samples.  [If the reason for this is not clear, I'd
be happy to follow up with an example]

	d.  As a follow-up to [c], what constraints would there be on
the "implicit" value for the Range header in a live stream where it is
reported as "npt=now-", assuming that "rtptime" clauses are present in
the RTP-Info header.  For example, could the server pick some point,
say, 5 minutes in the past as the synchronization point it will use to
generate the "rtptime" clause values?

	e. One of the most common current scenarios for distributing a
live broadcast is to have a "broadcaster" component that is *not* an
RTSP server -- it simply generates an RTP stream to a pre-defined set of
broadcast or unicast UDP ports and produces a ".sdp" file that describes
the broadcast.  The SDP file is staged in the media root of a full
RTSP/RTP streaming server.  When a client requests a session from the
".sdp", the server parses the *.sdp (which it may only become aware of
at the point of the first client request) to figure out where it should
"listen" for the streams (there is no RTSP dialog between the server and
the broadcaster).  In such a case, there is simply no way to guarantee
that the server knows *either* sequence number *or* synchronization
information for the streams before it needs to respond to the client.
In theory, this is disallowed by the new draft of the spec, is it not?
In practice, we (and others) have been using this method for years, and
it works just fine, so long as the client does not insist on having the
RTP-Info header.  The client simply "resyncs" itself when the first SRs
actually arrive in the data stream.  I realize that we're working on a
new mechanism  (the relay work) that would have the broadcaster be an
RTSP client and thus pass along the synchronization information when it
"announces" the availability of the presentation, but do you really
think we should be breaking an existing facility at the same time as we
are introducing its replacement?

	f. Two editorial notes on the draft:

		+ The separation of header syntax and sub-value
discussion (ala NPT discussion) from the header descriptions is very
confusing.  It makes it quite difficult to get the full story on any one
attribute of the specification (you end up having to simply search for
keywords in the document, which is complicated in the case of RTP-Info
because it shows up in several examples).

		+ Maybe I just don't know how to parse the syntax
properly, but it actually looks to me like the syntax asserts that the
"seq" and "rtptime" values are in fact mutually exclusive -- you can
only have one or the other (or is 1* supposed to be interpreted as "one
or more"?)

-- 
Andrew Large
[email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Magnus Westerlund
> Sent: Thursday, April 01, 2004 1:11 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: [MMUSIC] draft-rfc2326-06 comments wrt live streaming
> 
> 
> Hi Geetha,
> 
> See inline.
> 
> Geetha Srikantan wrote:
> > Magnus Westerlund wrote:

---

From: Geetha Srikantan <[email-masked]>
Date: Wed, 07 Apr 2004 16:53:40 -0700

hi Magnus,

Replying inline..

Magnus Westerlund wrote:

>> 1. The server may not see RTCP SRs for all streams in the RTSP 
>> session for several seconds,
>>   because of (a) staggered transmission of RTCP SRs of each stream, 
>> (b) lost SRs.
>>   Since RTCP transmission intervals can be 5-10 seconds, this could 
>> be 10-20 seconds or so.
>>
>> (a) deny all client requests for the live session until it receives 
>> RTCP SRs for all streams - even
>> If (a) and (b) are accepted as likely scenarios, then the clients 
>> cannot expect to see a RTP-Info header
Synchronization info is not always available when the origin/tier 1 RTSP 
server *starts* receiving
RTP streams from the live source. If the origin server is not to handle 
any requests for this live session,
then relay servers requesting this content will be stalled. That was my 
point.

I realize that when all servers have synchronization info available 
based on RTCP SRs or
RTP-Info/Range headers, a server can respond with the rtptime value for 
current wallclock time.

>> 2. The server may not know when the live session started, for example 
>> if it joins an ongoing
>>    live session.
>> One option is for the server to assume NPT=0 at the time it receives 
>> RTP packets from the
> information will only be present for the first user of a certain 
> stream. Any user requesting a stream already delivered to the delivery 
> server will be able to get correct information directly.

 From my experience, in live sessions, there are hundreds of clients 
trying to connect and receive the live session
within a few seconds of each other. So if the streaming server joins a 
live session and waits
to receive synchronization info before it responds to any client, all of 
these clients
would have to wait.
It's a tradeoff between (a) clients waiting for RTSP PLAY response, (b) 
clients receiving PLAY response without
synchronization info, decoding RTP streams right away and performing 
delayed synchronization.
Many of us have opted for (b) in implementations (as Andrew Large 
mentions in his email).

>> Reordering and lost packets make it difficult to populate the 
>> sequence number field of RTP-Info headers
>> for live streams. This is not the case for stored content. I think 
>> this is a point worth noting for live streaming.
>> If the server has to guess about the next packet it is going to send, 
>> it can't be a good thing.
>>
>
> joins one takes the last sequence number received. Then in the 
> RTP-Info one includes this number plus one. Then one forwards the next 
> packet, it this was reordered an is in fact an older packet than what 
> one claims in the RTP-Info, the client is able to detect that this 
> packet is prior to the start of the stream. If there is packet loss or 
> reorder that results in that the first packet that arrives to the 
> client has sequence number much higher than the indicated one, it is 
> okay. Because the only thing the RTP sequence number in the RTP-Info 
> indicates is from what sequence number the request media stream is 
> delivered.

I can see that this would work, however it seems like you are 
recommending that a server 'guess' about
the sequence number of the next packet it sends, since the server cannot 
accurately pinpoint the
exact sequence number of the next packet it sends. It might be 
preferable to interpret 'seqno ' as the
sequence number of the packet last sent by the server, so the client 
knows to look for sequence numbers higher than that.

> Magnus Westerlund

geetha

---

From: Geetha Srikantan <[email-masked]>
Date: Wed, 07 Apr 2004 17:13:36 -0700

hi Magnus,

Thanks for the pointer to the ABNF - yes it clarifies that RTP-Info can have
either seqno *or* rtptime.
I guess we agree that Range: npt=now- is acceptable in the PLAY response,
when accurate NPT value is not known.
My comment was about missing packets impacting seqno in the RTP-Info header
is covered in the previous email, so I'll skip it here.

Replying inline to the rest.

Magnus Westerlund wrote:

>> There is at least some media being rendered at the client, even if it 
>> is not lip-sync'd etc.
>>
>
> distributed using TCP, and will basically arrive as soon as possible. 
> If there are problems delivering the RTSP response.
>
> A server waiting for the synchronization information can also use a 
> 1xx code to tell that it is in fact waiting for the sync info.
>
> of the synchronization when the information arrives. Also without 
> giving the RTP-Info TS + Range it will be impossible to present 
> accurate session position information. A client will only be able to 
> calculate time since media delivery started.

As I mentioned in the previous email, it's a tradeoff between the client 
waiting for the RTSP handshake to complete and the client waiting for 
RTCP SRs, while it plays out individual streams. In the first case no 
media is rendered at the client until it completes the RTSP handshake 
and receives RTP.

> discussed in your streaming relay draft to achieve good performance 
> and avoid these issues you are so concerned with. From a client 
> perspective , I haven't seen a lot of need for clarifications. However 
> I can look on how to further improve the discussion in the RTP 
> appendix on synch. Otherwise I would recommend if you have specific 
> concerns to present proposals for new text.

We are working on a revision of the relay draft and will cover these 
issues there. However, I'd like to
point out that the live streaming issues do occur with or without 
relays, so they need to be addressed
in the main spec. Sure, I will work on proposing some text to clarify 
these issues and send it to the alias.

> Magnus
thanks
geetha

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 06 Apr 2004 16:59:18 +0200

> 	1. The RTP-Info header need have only the "seq" or "rtptime"

Yes, the header does not require both, however unless rtptime is used, 
synchronization is not achieved with this mechanism.

> Range header (e.g., when Range is "npt=now-").

Yes, however I have forgotten to point out that the case when using 
"npt=now-" will only achieve inter media synchronization when the media 
streams are under aggregated control.

Yes.

> know the sequence numbers of the first packets in each stream?

Yes, and this is the case for the first PLAY for a live stream to 
server. And in this case the lack of sequence number will not effect the 
client.

> number must differ by no more than N).

The point of sequence number in RTP-Info is to indicate the first RTP 
packet for which this synchronization information is valid. So it is not 
a matter of being able to be of with a certain amount. It is a matter of 
indicating from which packet the synch info is okay. So in a case where 
a server gives seq=C then the following observation can be done.

When the client receives an RTP packet with sequence number X the 
following applies:

if X<C:
This packet does not use the synchronization received in the RTP-Info + 
Range. Synchronization must be based on previous synch info. It might 
even be so that the most appropriate thing is to suppress this packet.

If X>=C:
This packet uses the synch info.

> be happy to follow up with an example]

 From draft version 6, chapter 11.4 PLAY

   "For a on-demand stream, the server MUST reply with the actual range
    that will be played back. This may differ from the requested range if
    alignment of the requested range to valid frame boundaries is
    required for the media source. If no range is specified in the
    request, the start position SHALL still be returned in the reply. If
    the medias that are part of an aggregate has different lengths, the
    PLAY request SHALL be performed as long as the given range is valid
    for any media, for example the longest media. Media will be sent
    whenever it is available for the given play-out point."

So I think it is clear that one should report what time interval one is 
actually playing. However it is not likely that one will be able to for 
all media stream send out RTP packets which matches that time exactly. 
Therefore the capability you are indicating is required to be supported 
by a client. However it might be necessary to further clarify this. An 
example like the following may be worth adding.

C->S:
PLAY
Range:npt=13.21-20

S->C
200 Ok
Range:npt=13.160-20
RTP-Info: url=audio;seq=50;rtptime=50000,url=video;seq=12;rtptime=123400

Then the RTP media arrives:

Audio (8kHz RTP timestamp rate):
S -> C: RTP packet - seq = 50, rtptime = 49830, NPT time = 13.15125 s

Video (90kHz RTP timestamp rate):
S -> C: RTP packet - seq = 10, rtptime = 1234000, NPT time= 13.160 s

So this indicate that it is expect that at least one media will have the 
same time as the indicate range in the response. However the client will 
need to handle that some media is prior than the requested and the 
indicated start time. What is currently not clear is if a server is 
allowed to send a audio packet with the following characteristics:

Audio (8kHz RTP timestamp rate):
S -> C: RTP packet - seq = 50, rtptime = 50100, NPT time = 13.1725 s

In other words a packet that is later than the indicated play time but 
prior to the request time.

Feedback on this is desired.

> generate the "rtptime" clause values?

In theory it could, but it would be less nice behaviour, but a client 
could experience this and should be capable of handling, because it is 
almost equal to a session where one losses 5 minutes of data. If you 
have the time 5 minutes ago, you only need to perform a multiplication + 
an addition per media to give a time value for the current time.

> are introducing its replacement?

Just to better understand the case: You are saying that the media root, 
is receiving the RTP stream from the broadcaster component, however it 
is totally ignoring it until a client requests it from the server?

If the server was not ignoring the stream, and instead parsed the RTCP 
information it would have the information.

Secondly, in the last revision we reduced the requirement on having 
RTP-Info from a MUST to a SHOULD, the current text says:

14.33 RTP-Info

    The RTP-Info response-header field is used to set RTP-specific
    parameters in the PLAY response. These parameters correspond to the
    synchronization source identified by the ssrc parameter of the
    Transport response header in the SETUP reponse. For streams using RTP
    as transport protocol the RTP-Info header SHOULD be part of a 200
    response to PLAY.

         The exclusion of the RTP-Info in a PLAY response for RTP
         transported media will result in that a client needs to
         synchronize the media streams using RTCP. This may have
         negative impact as the RTCP can be lost, and does not need
         to be particulary timely in their arrival. Also
         functionality as informing the client from which packet a
         seek has occurred is affected.

So the issue is simply if anyone is protesting against having this 
requirement on clients to handle media protocol (RTCP) for synch.

> because it shows up in several examples).

I think the main problem currently is that all headers description is 
lacking a bit of extra text to clarify syntax related questions. However 
I am not certain that I correctly understand you. Could you please 
better explain, and propose how to amend this problem.

> only have one or the other (or is 1* supposed to be interpreted as "one

1* = one or more.

Cheers

Magnus Westerlund

---

From: Andrew Large <[email-masked]>
Date: Thu, 08 Apr 2004 03:46:45 -0700

inline

-- 
Andrew Large
[email-masked]

> From: Magnus Westerlund [mailto:[email-masked]] 
> Sent: Tuesday, April 06, 2004 7:59 AM
> To: Andrew Large
> Cc: [email-masked]; [email-masked]
[snip]

> server. And in this case the lack of sequence number will not 
Okay, fair enough.  This can then probably be stated as "if you've got
it, you should send it; if you don't, don't worry about it".  One would
assume that a stream that was paused and resumed would have sequence
info.

> This packet uses the synch info.
So this sounds like "the seqno you send should be <= that of the first
packet the client receives from you".  So a client must be able to
handle the case where the first packet has a seqno that is maybe 2 or 3
values greater than what was put in the RTP-Info header, right?  If so,
that solves one of my concerns about the race condition implicit in
reflecting a live stream to several clients.

> RTP-Info: 
> url=audio;seq=50;rtptime=50000,url=video;seq=12;rtptime=123400
> 
I guess that, if I had to, I could live with the constraint that (for
"on demand" streaming) at least one 'rtptime' value must exactly equal
the value of the first packet sent in one of the streams.  I do feel
quite strongly that this must be the "last" of the first packets (I
think we're in agreement here).

However, consider a hypothetical presentation that has a single MIDI
stream.  The MIDI stream has a duration of 30 seconds, broken into three
samples (one packet per sample) with durations like this:

	packet 1: NPT=0.0 through 10.0
	packet 2: NPT=10.0 through 20.0
	packet 3: NPT=20.0 through 30.0

Now suppose that a user (via the client) requests a seek to NPT=7.0.

By my original proposal, I'd set the Range header to "7.0", but send
packet 1.  The data sample within the packet would have its own internal
timing information, which would let the client 'skip' the first 7
seconds of the MIDI sample and start playing at npt=7.0.

With your constraint, I'd be forced to put either "0.0" or "10.0" in my
Range header (depending upon which packet I selected).  I submit that
neither of these would be very palatable from an end-user perspective.

Hopefully, its clear that I'm not arguing for an ability to start
streams at an arbitrary point earlier than the "Range" response.
Rather, I'm suggesting that the deviation from the Range header on *all*
tracks could be as much as one media sample's duration.

Just to be clear -- my example isn't all that contrived.  We've
definitely seen/streamed MIDI tracks, and there are efforts underway to
develop "timed text" specifications which may very well have the same
"long duration sample" properties.

> an addition per media to give a time value for the current time.
Right, so this was really a follow-up to the above discussion.  I'm very
much in favor of insisting (for "on demand" streams) that the value in
the 'Range' header be >= the RTP timestamps of the first packets sent by
the server in all streams.  This gives a guarantee that a client has all
the media necessary to begin the presentation at that NPT.

However, in the case of a live stream, we aren't really saying anything
in the Range header about where the 'presentation start' point is -- nor
does a user/client select such a point.  Everything is basically "now"
(pause now, play now).  Hence I find it difficult to justify any sort of
required relationship between the 'RTP-Info' header 'rtptime' values and
the first RTP timestamps the client receives (except the obvious that
conversions from RTP timestamp to/from NPT must generate correct
answers).  This is one of my issues with requiring the RTP-Info header
for live streams -- I don't like having two different modes of
interpretation -- one for "on demand", and the other for "live".

> information it would have the information.
Almost, but not quite:

	* Broadcaster on host A is sending streams to host B, ports I
and J.  The individual who sets up the broadcast manually transfers a
".sdp" file into the content root on host B.

	* Server on host B has no idea that the broadcaster exists and
has no idea that it has been given a new ".sdp" file.

	* Client requests the ".sdp" file

	* In servicing the request, server notes that it isn't currently
receiving this broadcast and sets up listeners on ports I and J.

This is the way most live streaming we have seen is actually done today.

The notion that the server would hold up the PLAY reply until it
receives RTCP SRs is problematic in a ton of ways.  There is no
guaranteed upper bound on this wait period.  How long do you wait before
you time out and reject the request?  And what exactly are you gaining
by waiting?  You are making an end-user sit for (potentially) several
seconds before media buffering even begins.  How much do you lose by
simply omitting the RTP-Info header and beginning immediate packet
forwarding?  Consider two cases:

	* SRs come very soon: user will have very short period of
"slightly out of sync" presentation -- in fact, maybe none at all
(because the client may be able to process the sync while still doing
initial buffering).

	* SRs come after several secs: user will have long period of
"out of sync" presentation before the receipt of the SRs syncs things
up.  However, if you force the wait, the user will simply spend those
same seconds looking at a blank screen (and they'll never see that part
of the presentation).

> requirement on clients to handle media protocol (RTCP) for synch.
I guess my issue is that I think we need to bite the bullet and be clear
that the requirements are simply different for live streams and "on
demand" streams:

	* For reflected live streams, you may not be able to include
SSRC in SETUP response (because you haven't seen any packets); for "on
demand", you can.

	* For reflected live streams, the Range header is useless,
except as a mechanism to signal that this *is* a live stream (e.g.,
npt=now-).  For "on demand" streams, it is critical.

	* For reflected live streams, the RTP-Info header may simply not
be there (and maybe even *shouldn't* be there); if it is there, it is
interpreted a little differently than with "on demand" streams.  For "on
demand" streams, it should be there, fully populated, and deeply
understood.

> better explain, and propose how to amend this problem.
Maybe an example would clarify this.  I went to section 14.29 expecting
to get the full definition of the Range header and how it is used.  What
I did *not* find in 14.29:

	* The different ways of specifying the range (had to go to
sections 3.4, 3.5, and 3.6).

	* The "live streaming" correlation with "npt=now-" (section
11.4)

	* The actual syntax for the Range header (subtly buried in
section 17.2.3)

So what would be my proposal?  Well, to define/describe things (like
headers) in exactly one place in the spec and not split and distribute
definitions, e.g.:

	* Put the syntax with the header description (e.g. ditch 17.2.3
and pull it into the header descriptions)

	* Pull in all the context-related usage that is scattered
through other parts of the spec (suck 3.4, 3.5, 3.6 and relevant text in
11.4 into 14.29).

Now, I realize this is probably more than you're willing to bite off at
this point, but maybe there are some "low hanging fruit" things you
might think of doing if you recognize and agree with the source of my
frustration.

BTW, a little better description of when you use what error codes would
definitely be helpful.  For example, what error should a server return
if it attempts to parse a file requested for an "on demand" presentation
and finds that the file is corrupted in some fashion?

> > only have one or the other (or is 1* supposed to be 
Yeah, figured that one out after spending a little more time here :-)