Subject: [MMUSIC] RTSP: Speed header

From: Magnus Westerlund <[email-masked]>
Date: Mon, 16 Apr 2007 16:27:47 +0200

Hi,

I while back received an email with some questions on how the Speed 
header works in RTSP. There where also the issue how it interacts with 
the Scale header and with bandwidth adaptation and buffer management.

Lets do a recap of what the different mechanisms do.

Scale: Changes the rate at which media is presented to the user. So 
Scale=0.5 would mean slow motion, while Scale=5 would be fast forward in 
the stream.

Speed: Changes the media delivery rate. Speed=2 would mean that for each 
second a client would receive media that takes it 2 seconds to play back 
to the user. It would also mean that using RTP with a 90kHz timestamp 
clock the RTP streams timestamp values would increase with 180k ticks 
per second.

Bit-rate adaptation: A way of ensuring that the media traffic does not 
use more bandwidth then what is available. Which means that the server 
would need to change the media bit-rate sent if there isn't enough 
available for the current media stream. This could be done in a number 
of ways but most commonly are done by bit-stream switching. Going from a 
stored/delivered media stream in one quality to another lower quality 
that requires less bit-rate.

Buffer Management: An attempt to keep the clients buffer in a healthy 
state, aiming at keeping a certain amount of media time present in 
buffer to safeguard against temporary or longer term changes of bit-rate 
capacity available to the streaming session. Thus giving the server time 
to react to the change before an buffer-underrun situation occurs. At 
the same time safe-guard against overflowing buffers. This is for 
example done in 3GPP Packet-based Streaming Service (26.234 Rel6) where 
additional feedback and configuration information is provided for the 
buffer status. Thus allowing a server to change media transmission rate 
on the fly. Which is equivalent to a client using Speed. However it also 
needs to be able to perform bit-rate adaptation so that one can lower 
the quality but increase the speed to refill the buffer.

The interactions between the different behaviors are interesting but can 
become quite complex. For example the total bit-rate is speed x normal 
bit-rate. It might also be influenced by the Scale used as fast 
forwarding could be accomplished by simple turning up the frame-rate for 
video and rewriting the timestamps compared to scale 1. Slow motion 
could also mean that the bit-rate gets lower. However if trans-coding is 
done to handle Scale then bit-rate may not change at all. If there are a 
limited amount of bit-rate this means that bit-rate adaptation needs to 
step in and do what it can to keep bit-rate within limitations. If that 
can only operate on the media stream or can also contradict speed or 
influence how scale is handled is quite undefined.

I do have a few question due to this:

1. Are people implementing Speed?

2. If you are using Speed, how are you doing that, as I have described 
or otherwise?

3. Should we remove speed from base specification?

4. If not, what do you think needs to be clarified and do you have text 
proposal?

Cheers

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM/M
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

_______________________________________________
mmusic mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mmusic

---

From: Dave Singer <[email-masked]>
Date: Mon, 23 Apr 2007 10:12:15 -0700

I have to confess I find both the existing text and what you write 
below quite confusing. e.g.  If I am trying to play back at normal 
rate, why would I ask for the data to be delivered at say half or 
double rate?

I think I read it that scale (roughly) tells the server what rate I 
want to play at, and speed seems to have an effect on the bandwidth 
the server can use.

I know of no-one implementing these, partly because, I think, that 
there is some doubt over the meaning.  I have to say that I find even 
the names rather odd!

So, I would read, perhaps?
scale=2, speed=1 -- send it to me for double-speed playback, but 
don't use any more bandwidth than normal
scale=2, speed=2 -- you can also use twice the normal bandwidth
scale=-1, speed=1 -- send to me in normal speed reverse order

I'm not sure.  Can someone give us some background?
-- 
David Singer
Apple Computer/QuickTime

---

From: Ross Finlayson <[email-masked]>
Date: Mon, 23 Apr 2007 19:23:52 -0700

One of the strengths of the IETF is that we prefer not to standardize 
protocol features that are not being used.  With that in mind, 
perhaps the "Speed" header should be removed from the RTSP 2.0 
specification, especially since - as Dave makes clear - its purpose 
appears unclear.

(I had always assumed that this header was intended to be used for 
'fast downloading' - e.g., downloading a 1 hour movie in 30 minutes 
(assuming sufficient bandwidth).  But if that's what you want, then 
why not just download the movie as a file?)

Do we have any reason to keep the "Speed" header in the RTSP 2.0 specification?

	Ross.

---

From: Henning Schulzrinne <[email-masked]>
Date: Mon, 23 Apr 2007 22:33:34 -0400

The original motivation was to have a more controlled, but higher- 
bandwidth, download, particularly in environments where TCP was  
inappropriate (say, multicast).

That said, if there's no current usage of that feature, it should be  
relegated to an extension.

On Apr 23, 2007, at 10:23 PM, Ross Finlayson wrote:

> One of the strengths of the IETF is that we prefer not to  
> standardize protocol features that are not being used.  With that  
> in mind, perhaps the "Speed" header should be removed from the RTSP  
> 2.0 specification, especially since - as Dave makes clear - its  
> purpose appears unclear.
>

---

From: "Ingemar Johansson S \(LU/EAB\)" <[email-masked]>
Date: Tue, 24 Apr 2007 08:44:57 +0200

Hi

Reading this makes me think of many of the sites out there that allows
the user watch video clips, to me it seems like the download speed is
faster than the actual playout speed. 
One such example is YouTube, the benefit is that one can quite easily
fastforward in the videoclip with very little extra delay. 
Cannot tell if it is implemented as streaming or file download or how
YouTube works on a technical level but anyway the functionality as seen
by the user is there.
 
Ingemar

> -----Original Message-----
> From: Henning Schulzrinne [mailto:[email-masked]] 
> Sent: den 24 april 2007 04:34
> To: Ross Finlayson
> Cc: [email-masked]
> Subject: Re: [MMUSIC] RTSP: Speed header
> 
> The original motivation was to have a more controlled, but 
> higher- bandwidth, download, particularly in environments 
> where TCP was inappropriate (say, multicast).
> 
> That said, if there's no current usage of that feature, it 
> should be relegated to an extension.
> 
> standardize 
> > protocol features that are not being used.  With that in 
> mind, perhaps 
> > the "Speed" header should be removed from the RTSP 2.0 
> specification, 
> > especially since - as Dave makes clear - its purpose 
> appears unclear.
> >

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 24 Apr 2007 10:03:31 +0200

> YouTube works on a technical level but anyway the functionality as seen

downloading at slightly faster than media speed or even at full fair 
share is fine for media file. However for streaming there are to some 
degree not necessary to buffer enormous amounts because you loose the 
interactivity, or at least you will be forced to flush you buffer at 
time you change your playback point or so.

Lets wait and see a few more days, but so far there seem to be little 
real interest for speed. Especially its problematic relation to 
congestion control makes it difficult to use. Because the request can 
only be fulfilled if the actual bandwidth is available.

Cheers

Magnus Westerlund

---

From: "Heiles, Juergen" <[email-masked]>
Date: Tue, 24 Apr 2007 10:43:42 +0200

Such Internet video sides usually use http based Progressive Download
instead of streaming.
See http://www.streamingmedia.com/article.asp?id=8456&page=1 for the
difference.

Regards

Juergen

>From: ext Ingemar Johansson S (LU/EAB) 
>[mailto:[email-masked]] 
>Sent: Tuesday, April 24, 2007 8:45 AM
>To: Henning Schulzrinne; Ross Finlayson
>Cc: [email-masked]
>Subject: RE: [MMUSIC] RTSP: Speed header
>
>Hi
>

---

From: Dave Singer <[email-masked]>
Date: Tue, 24 Apr 2007 10:39:45 -0700

I agree that if you want download, at the rate that the 
network/server/client can support, a, um, download protocol is 
generally better.  TCP provides all the flow control and so on.

I do think that an RTSP 2.0 should clarify trick modes.  I don't 
think that download needs to be in there, but data-rate management 
does need to be.
-- 
David Singer
Apple Computer/QuickTime

---

From: Aaron Colwell <[email-masked]>
Date: Tue, 24 Apr 2007 12:55:07 -0700

We've used something similar to Speed in the Real/HelixServer for client 
controlled rate adaptation. The client used our "Speed" equivalent to 
slightly increase the delivery rate to see if there was bandwidth to 
switch up to the next higher bitrate. Depending on how often it 
increases the rate above the clip rate, the client may also need to 
lower the delivery rate below the clip rate to prevent buffering too 
much data. We do not switch the RTP sample rate when we increase the 
delivery rate like Magnus' original email suggests.

I'm not making a case whether Speed should stay or not, just providing 
an example of where such functionality has been used in an existing 
implementation.

Aaron

Dave Singer wrote:
> I have to confess I find both the existing text and what you write below 
> quite confusing. e.g.  If I am trying to play back at normal rate, why 
> would I ask for the data to be delivered at say half or double rate?
> 
> I think I read it that scale (roughly) tells the server what rate I want 
> to play at, and speed seems to have an effect on the bandwidth the 
> server can use.
> 
> I know of no-one implementing these, partly because, I think, that there 
> is some doubt over the meaning.  I have to say that I find even the 
> names rather odd!
> 
> 
> scale=2, speed=1 -- send it to me for double-speed playback, but don't 
> use any more bandwidth than normal

---

From: sunil bhargo <[email-masked]>
Date: Tue, 24 Apr 2007 19:28:20 -0700 (PDT)

Though there are other methods but if the
client wants to initially start the playback
quickly/fill the initial buffers faster then Scale=1
and Speed=2 (greater than bitrate) can be used.
          Also these headers can help a proxy(case of
reverse proxy) to maintain the connections to the
server for a lesser duration of time as it can ask the
server to stream the data at a higher rate and then
satisfy the request locally.
          These headers can help though basically only
if the player/client has a way to intelligently detect
the available bandwidth either through some
packet-pair algorithm or some equivalent of that. Most
of the clients i think have a variation of these
headers if not the same headers.

regards,
sunil bhargo

--- Magnus Westerlund <[email-masked]>

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

---

From: Darren <[email-masked]>
Date: Wed, 25 Apr 2007 17:00:45 -0600

We have implemented a scale capability and bit rate adaptation ability on
top of RTSP 1.0.  It would be great to have something standard in that area.

Regarding "speed", one can improve the user experience when fast-forwarding
and fast-rewinding video when there is a lot of bandwidth available, such as
on a local LAN.

ie: 150Kbps stream sent at 2 or 3x fast forward looks really good if the
150Kbps stream is sent at 750Kbps.  It allows a user to more easily scan and
find a specific moment in a video stream.

Having "speed" would allow one to specify exactly the behavior desired.  But
I'm not sure it's required given the other capabilities out there.

-Darren

---

From: "Ingemar Johansson S \(LU/EAB\)" <[email-masked]>
Date: Thu, 26 Apr 2007 09:02:51 +0200

I too beleive that from a user perspective "speed" can be quite useful.
80 year old grandma probably has all time in the world to watch an
entire news clip while the "busy bee" probaly only have 20 minutes in
the cummuter train to watch the same 30 minute news clip, and therefore
wish to fastforward though the news headline about the flower
exhibition.
 
One argument I read is that it is better then to use (progressive)
download in the latter case instead but the result is then that
depending on e.g the user behavior you will have to switch between
protocols. To me it is better to use the same protocol (not a firm
though though as there is always a possibility that the "busy bee" has
has already subscribed to some download service instead). 
 
Also, as Magnus pointed out there is always the possibility to trade
bitrate for speed if bandwidth is scarce. It may of course be complex
interactions but if the mechanisms exists the possibilities are there.
 
Ingemar

________________________________

	From: Darren [mailto:[email-masked]] 
	Sent: den 26 april 2007 01:01
	To: Magnus Westerlund (KI/EAB)
	Cc: mmusic (E-mail)
	Subject: Re: [MMUSIC] RTSP: Speed header

	We have implemented a scale capability and bit rate adaptation
ability on top of RTSP 1.0.  It would be great to have something
standard in that area.
	
	Regarding "speed", one can improve the user experience when
fast-forwarding and fast-rewinding video when there is a lot of
bandwidth available, such as on a local LAN. 
	
	ie: 150Kbps stream sent at 2 or 3x fast forward looks really
good if the 150Kbps stream is sent at 750Kbps.  It allows a user to more
easily scan and find a specific moment in a video stream. 
	
	Having "speed" would allow one to specify exactly the behavior
desired.  But I'm not sure it's required given the other capabilities
out there. 
	
	-Darren

	On 4/16/07, Magnus Westerlund <[email-masked]>
wrote: 

		Hi,
		
		I while back received an email with some questions on
how the Speed
		header works in RTSP. There where also the issue how it
interacts with
		the Scale header and with bandwidth adaptation and
buffer management. 

		Scale: Changes the rate at which media is presented to
the user. So
		Scale=0.5 would mean slow motion, while Scale=5 would be
fast forward in
		the stream. 
		
		Speed: Changes the media delivery rate. Speed=2 would
mean that for each
		second a client would receive media that takes it 2
seconds to play back
		to the user. It would also mean that using RTP with a
90kHz timestamp 
		clock the RTP streams timestamp values would increase
with 180k ticks
		per second.
		
		Bit-rate adaptation: A way of ensuring that the media
traffic does not
		use more bandwidth then what is available. Which means
that the server 
		would need to change the media bit-rate sent if there
isn't enough
		available for the current media stream. This could be
done in a number
		of ways but most commonly are done by bit-stream
switching. Going from a 
		stored/delivered media stream in one quality to another
lower quality
		that requires less bit-rate.
		
		Buffer Management: An attempt to keep the clients buffer
in a healthy
		state, aiming at keeping a certain amount of media time
present in 
		buffer to safeguard against temporary or longer term
changes of bit-rate
		capacity available to the streaming session. Thus giving
the server time
		to react to the change before an buffer-underrun
situation occurs. At 
		the same time safe-guard against overflowing buffers.
This is for
		example done in 3GPP Packet-based Streaming Service
(26.234 Rel6) where
		additional feedback and configuration information is
provided for the
		buffer status. Thus allowing a server to change media
transmission rate
		on the fly. Which is equivalent to a client using Speed.
However it also
		needs to be able to perform bit-rate adaptation so that
one can lower
		the quality but increase the speed to refill the buffer.
		
		The interactions between the different behaviors are
interesting but can
		become quite complex. For example the total bit-rate is
speed x normal
		bit-rate. It might also be influenced by the Scale used
as fast 
		forwarding could be accomplished by simple turning up
the frame-rate for
		video and rewriting the timestamps compared to scale 1.
Slow motion
		could also mean that the bit-rate gets lower. However if
trans-coding is 
		done to handle Scale then bit-rate may not change at
all. If there are a
		limited amount of bit-rate this means that bit-rate
adaptation needs to
		step in and do what it can to keep bit-rate within
limitations. If that 
		can only operate on the media stream or can also
contradict speed or
		influence how scale is handled is quite undefined.

		
		2. If you are using Speed, how are you doing that, as I
have described 
		or otherwise?

		4. If not, what do you think needs to be clarified and
do you have text
		proposal?
		
		Cheers
		
		Magnus Westerlund

	
----------------------------------------------------------------------
		Ericsson AB                | Phone +46 8 4048287 
		Torshamsgatan 23           | Fax   +46 8 7575550
		S-164 80 Stockholm, Sweden | mailto:
[email-masked]

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 26 Apr 2007 14:02:51 +0200

Ingemar Johansson S (LU/EAB) skrev:
> wish to fastforward though the news headline about the flower exhibition.

But for that you don't need speed. you need scale unless you actually 
like to buffer and do the speedup locally. Which will consume more 
decoding resources etc.

> has already subscribed to some download service instead). 

There are some cases where progressive download is a simple solution to 
the problem. However, progressive download are in many cases lousy for 
fast forward, jumping etc unless you download the file very fast and 
then do all these tricks locally.

When you like to utilize available bit-rate fully and still be able to 
manipulate the playback then RTSP has some great benefits.

> interactions but if the mechanisms exists the possibilities are there.

As I see it the problem with speed is how one use it. A client can use 
it for buffer management. Another usage is related to flow/congestion 
control. But that I think second is definitely the wrong decision. We 
need have congestion control of the media stream sent from RTSP. For all 
protocols that can be used. This would be much easier if RTP over DCCP 
was commonly available. If one runs RTP over TCP the congestion control 
part is well taken care of, but one loose some of the benefits of RTP 
over datagrams.

I think we need to split this discussion into two topics:

a) How to solve client buffer management?

b) How to solve adaptation to available bit-rate

And in relation to both these topics have discussion on how one uses 
Scale=x where X!=1.0

And when we actually agree on how to do the different topics then we can 
see if the RTSP Speed header is still part of the discussion.

Cheers

Magnus Westerlund