Subject: Re: [AVT] SVC open issues - sprop-spatial-resolution

From: <[email-masked]>
Date: Fri, 30 May 2008 11:12:42 +0300

I have studied earlier email threads (including Roni's suggestion in
2002) Randell mentioned below, as well as the following related
documents:
- The a=framesize defined in 3GPP PSS
(http://www.3gpp.org/ftp/Specs/archive/26_series/26.234/26234-750.zip)
- The newly consented ITU-T H.241video submode negotiation (in ITU-T
document T05-SG16-080422-TD-PLEN-0490!!ZIP-E.zip)
- The 3GPP Feature or Study Item: MTSI Video - Dynamic Rate
Adaptation/Signalling of Image Size
(http://www.3gpp.org/ftp/Specs/html-info/GanttChart-Level-2.htm#bm34041)
, and the input document S4-080145
(http://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/TSGS4_48/Docs/S4-080145.zip)
 
Summary:
- The purpose of having the sprop-spatial-resolution parameter for SVC
use is different from any other earlier-mentioned purposes. Herein, the
parameter is only used to indicate the property of an SVC bitstream or a
subset thereof, and may be used in negotiation of a subset of an SVC
bitstream to be transmited from the sender to the receiver. 
- The sprop-spatial-resolution parameter is closest, in semantics, to
the PSS a=framesize, which is defined as "a=framesize:<payload type
number> <width>-<height>". Having <payload type number> is not needed
for the SVC use, but it may not harm.

Example:
-  In unicast streaming, the client gets an SDP (e.g. by sending RTSP
DESCRIBE method to the server and sever sends back the SDP) for an SVC
bitstream containing multiple operation points, each being assoicated
with a spatial resolution, frame rate, and bit rate, among others. The
client then choose one of the operation points and tells the server its
choice. Then the server sends the bitstream subset for the chosen
operation point. 

Conclusion: 
- An SDP parameter having similar semantics as sprop-spatial-resolution
or the PSS a=framesize is needed. 

The way to go:
- Either to have a generic SDP parameter defined somewhere else or to
have a SVC specific defintion as it is. If there is no way to do the
former quickly, then let's leave the definition in the SVC draft. 

BR, YK

>-----Original Message-----
>From: ext Randell Jesup [mailto:[email-masked]] 
>Sent: 14 April, 2008 17:33
>To: Wang Ye-Kui (Nokia-NRC/Tampere)
>Cc: [email-masked]; [email-masked]
>Subject: Re: [AVT] SVC open issues - sprop-spatial-resolution
>
>Everyone interested in this discussion should review the last
>time we discussed this in 2004 (note - the thread index links 
>seem to take
>you to 2007; prev and next work):
>
>http://www.ietf.org/mail-archive/web/mmusic/current/msg03137.html
>and
>http://www.ietf.org/mail-archive/web/avt/current/msg04871.html
>
>Note: the discussion goes on for a while.
>
>Also, Roni's suggestion from 2002:
>http://www.ietf.org/mail-archive/web/mmusic/current/msg00649.html
>
>Also, from RFC 4566:
>      a=framerate:<frame rate>
>
>         This gives the maximum video frame rate in frames/sec.  It is
>         intended as a recommendation for the encoding of video data.
>         Decimal representations of fractional values using 
>the notation
>         "<integer>.<fraction>" are allowed.  It is a media-level
>         attribute, defined only for video media, and it is not
>         dependent on charset.
>
>Note that this "is intended as a recommendation for the encoding".
>I.e. that does not say "you cannot encode at a higher rate"; 
>it says "I'd
>like you to encode at this rate (or lower)".  Also, this is 
>media-level, as
>it says, so it doesn't help with codec-specific issues; it 
>maps better to
>output-device ability (or ability of whatever codec offered is 
>fastest).
>
>"Even, Roni" <[email-masked]> writes:
>>I am not sure what the requirements for this parameter are.
>>
>>There is no generic resolution parameter defined. There are 
>specific ones
>>in H.263 and H.261 and they list the supported resolution in 
>relations to
>>the frame rate. 
>
>A generic solution would be better, or a specific solution 
>that could be
>generalized later.  See previous discussions.  I should note 
>some devices
>use weird resolutions, like 352x240 (kinda 1/2 CIF, 1/2 QVGA) 
>or unusual
>layouts like 144x176 (portrait mode).  :-)
>
>>In RFC 3984 the receiver capabilities max-mbps and max-fs allows the
>>receiver to signal that it supports higher macro blocks per second or
>>frame size in macro blocks than the signaled level according to table
>>A-1. 
>
>Those can define decoder ability, but not display ability.  Note that
>a level 3 decoder could decode QCIF (99 macroblocks) at very high frame
>rates, but typically could not display it any faster than the 
>refresh rate
>of the physical display device (30/50/60Hz, typically), so higher frame
>rates would be wasted.
>
>Maximum display ability is better specified by a=framerate, 
>per above, on a
>media level.
>
>I should note that a device which can support max-fs of X and 
>max-mbps of Y
>cannot necessarily support them at the same time.  At max-fs, 
>the decoder
>might need to use an expensive output scaler, or to decode a maximum
>framesize it might have to kick out some data from cache, hurting the
>max-mbps.  (For example, at SQCIF the current frame and 
>reference frame(s)
>might all fit in L2 or L3 cache, enabling a very high 
>max-mbps, while at
>VGA they may not fit, slowing it down a lot.)
>
>How much that matters in practice is another question - I suspect most
>don't run that close to the limit.
>
><[email-masked]> writes:
>>Let me try to explain with an example. Assume that a streaming server
>>offers a scalable bitstream containing three operation points with
>>different resolutions, e.g. QCIF, CIF and 4CIF. In the SDP, it should
>>include multiple m lines, each operation point should be 
>associated with
>>one m line. Layer dependency is inidcated according to the MMUSIC
>>decoding dependency draft. Beside layer dependency 
>information, for each
>>of the operation point, other important information such as bitrate,
>>frame rate and spatial resolution should also be included. My poor
>>understanding to SDP is that it provides means to indicated bitrate,
>>frame rate but not spatial resolution - and I don't why SDP does not
>>include this parameter.
>
>It's been discussed (per above), but never handled on a generic level.
>
>>With RFC 3984 it is possible to indicate the spatial resolution of a
>>stream using the sequence parameter set included in one OPTIONAL media
>>type parameter sprop-parameter-sets, but there is no other means as I
>>know. However, in SVC, this sprop-parameter-sets for the 4CIF 
>stream in
>>the example above may contain sequence parameters associated with
>>different spatial resolutions. Furthermore, it is not 
>convenient to dig
>>out the spatial resolution values buried in the variable-length coded
>>parameter sets in sprop-parameter-sets. 
>
>Exactly; parsing them out of the parameter sets is a non-starter.
> 
>-- 
>Randell Jesup, Worldgate (developers of the Ojo videophone), 
>ex-Amiga OS team
>[email-masked]
>"The fetters imposed on liberty at home have ever been forged 
>out of the weapons
>provided for defence against real, pretended, or imaginary 
>dangers from abroad."
>		- James Madison, 4th US president (1751-1836)
>
>
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt

---

From: "Ingemar Johansson S" <[email-masked]>
Date: Sun, 1 Jun 2008 10:51:54 +0200

Hi
FYI I submitted an "a=imageattr" draft to MMUSIC
as a response to input document S4-080145
(http://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/TSGS4_48/Docs/S4-080145.zip)

The link to the draft is 
http://tools.ietf.org/id/draft-johansson-mmusic-image-attributes-00.txt
================
Abstract

   This draft proposes a new generic session setup attribute to make it
   possible to negotiate different image attributes such as image size.
   A possible use case is to make it possible for a e.g a low-end hand-
   held terminal to display video without the need to resample the
   image, something that may consume large amounts of memory and
   processing power.
================

Regards
Ingemar

> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of [email-masked]
> Sent: den 30 maj 2008 21:00
> To: [email-masked]
> Subject: avt Digest, Vol 49, Issue 20
> 
> Send avt mailing list submissions to
> 	[email-masked]
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> or, via email, send a message with subject or body 'help' to
> 	[email-masked]
> 
> You can reach the person managing the list at
> 	[email-masked]
> 
> When replying, please edit your Subject line so it is more 
> specific than "Re: Contents of avt digest..."
> 
> 
> Today's Topics:
> 
>    1. Re: SVC open issues - sprop-spatial-resolution
>       ([email-masked])
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Fri, 30 May 2008 11:12:42 +0300
> From: <[email-masked]>
> To: <[email-masked]>, <[email-masked]>
> Cc: [email-masked], [email-masked]
> Message-ID:
> 	<[email-masked]>
> Content-Type: text/plain;	charset="us-ascii"
> 
>  
> - The newly consented ITU-T H.241video submode negotiation 
> (in ITU-T document T05-SG16-080422-TD-PLEN-0490!!ZIP-E.zip)
> - The purpose of having the sprop-spatial-resolution 
> parameter for SVC use is different from any other 
> earlier-mentioned purposes. Herein, the parameter is only 
> used to indicate the property of an SVC bitstream or a subset 
> thereof, and may be used in negotiation of a subset of an SVC 
> - The sprop-spatial-resolution parameter is closest, in 
> semantics, to the PSS a=framesize, which is defined as 
> "a=framesize:<payload type
> -  In unicast streaming, the client gets an SDP (e.g. by 
> sending RTSP DESCRIBE method to the server and sever sends 
> back the SDP) for an SVC bitstream containing multiple 
> operation points, each being assoicated with a spatial 
> resolution, frame rate, and bit rate, among others. The 
> client then choose one of the operation points and tells the 
> server its choice. Then the server sends the bitstream subset 
> for the chosen operation point. 
> 
> Conclusion: 
> - An SDP parameter having similar semantics as 
> sprop-spatial-resolution or the PSS a=framesize is needed. 
> 
> The way to go:
> - Either to have a generic SDP parameter defined somewhere 
> else or to have a SVC specific defintion as it is. If there 
> is no way to do the former quickly, then let's leave the 
> definition in the SVC draft. 
> 
> BR, YK
> 
> 
> >Everyone interested in this discussion should review the 
> last time we 
> >discussed this in 2004 (note - the thread index links seem 
> to take you 
> >to 2007; prev and next work):
> >
> >         Decimal representations of fractional values using the 
> >notation
> >I.e. that does not say "you cannot encode at a higher rate"; it says 
> >"I'd like you to encode at this rate (or lower)".  Also, this is 
> >media-level, as it says, so it doesn't help with 
> codec-specific issues; 
> >it maps better to output-device ability (or ability of 
> whatever codec 
> >offered is fastest).
> >
> that could 
> >be generalized later.  See previous discussions.  I should note some 
> >devices use weird resolutions, like 352x240 (kinda 1/2 CIF, 
> 1/2 QVGA) 
> >or unusual layouts like 144x176 (portrait mode).  :-)
> >
> >Those can define decoder ability, but not display ability.  
> Note that a 
> >level 3 decoder could decode QCIF (99 macroblocks) at very 
> high frame 
> >rates, but typically could not display it any faster than 
> the refresh 
> >rate of the physical display device (30/50/60Hz, typically), 
> so higher 
> >frame rates would be wasted.
> >
> per above, 
> >on a media level.
> >I should note that a device which can support max-fs of X 
> and max-mbps 
> >of Y cannot necessarily support them at the same time.  At 
> max-fs, the 
> >decoder might need to use an expensive output scaler, or to decode a 
> >maximum framesize it might have to kick out some data from cache, 
> >hurting the max-mbps.  (For example, at SQCIF the current frame and 
> >reference frame(s) might all fit in L2 or L3 cache, enabling a very 
> >high max-mbps, while at VGA they may not fit, slowing it down a lot.)
> >
> ex-Amiga 
> >OS team [email-masked] "The fetters imposed on liberty at 
> home have 
> >ever been forged out of the weapons provided for defence 
> against real, 
> >pretended, or imaginary dangers from abroad."
> ------------------------------
> 
> End of avt Digest, Vol 49, Issue 20
> ***********************************
>