Subject: [MMUSIC] Re: Maximum non-scaled image size

From: Aron Rosenberg <[email-masked]>
Date: Tue, 22 Mar 2005 10:25:05 -0800

The 3GPP ID already defines most of the attributes that you are looking 
for: draft-westerlund-mmusic-3gpp-sdp-rtsp-00.txt

"a=framesize:<payload type number> <width>-<height>"
"a=3gpp-videopostdecbufsize"

For framerate see the MPI attribute in the H.263 / RTP / SDP spec: 
draft-ietf-avt-rfc2429-bis-04.txt

SightSpeed uses these to support proper Video identification and sizes 
over a broad range of devices. When combined with the offer/answer 
model, the receiver can modify the attributes to indicate lesser support 
of a given attribute. So long as the sender checks the answer, you can 
negotiate a proper media session. Most everything that you are looking 
for is already in some sort of proposed form. There is no need for 
separate Display rates or codec rates, the end device determines its 
lowest common rate and highest rate and offer/answers that.

Aron Rosenberg
SightSpeed - Video Phone Service that Works <http://www.sightspeed.com>

>
>----------------------------------------------------------------------
>
>Message: 1
>Date: 21 Mar 2005 16:25:18 -0500
>From: Randell Jesup <[email-masked]>
>Subject: Re: [MMUSIC] Maximum non-scaled image size
>To: [email-masked]
>Message-ID: <[email-masked]>
>Content-Type: text/plain; charset=us-ascii
>
>  
>
>>>>Based on the following Sentence:
>>>>"Do any video payload formats specify, as SDP format attributes or
>>>>otherwise, the maximum image size supported natively (i.e., without
>>>>downscaling)?"
>>>>        
>>>>
>
>  
>
>>>It talks about native support of image size, which I interpreted to mean
>>>actual screen size. As the actual amount of macroblock, size of frame
>>>buffer, that a receiver is capable of decoding is different from what it
>>>natively can display. Thus I think there is clear reasons to separate the
>>>capability for decoding from that of displaying. However I don't
>>>necessarily think that it is appropriate to consider display resolution
>>>capabilities in SDP, and definitely not in any overlapping
>>>parameter. There are good reasons to differentiate the two capabilities.
>>>      
>>>
>
>  
>
>>Fully agreed. Among other differences, the display size is likely a "soft"
>>limit, i.e., something that the sender can exceed if you don't care about
>>wasting bits. The frame buffer size seems more of a hard limit.
>>    
>>
>
>        With small/embedded devices (cellphones, etc), you can't always
>just go over a natural limit and have things be usable (even if the
>programmer wanted to).  Scalers are not cheap if there isn't dedicated
>hardware; going over the display resolution may well not scale but instead
>may crop (and the crop may be upper, left, middle, who knows?).  It may not
>work at all (even if the decode HW has the capability to decode the number
>of macroblocks per second you're sending) due to buffer sizes or other hard
>or soft limits in the codec implementation.
>
>        Similar things exist with going under preferred resolutions; a
>device may not have a scaler, or if it doesn it may well not be arbitrary.
>
>        As you say, there may be frame buffer or decoder limits (and not
>just in bytes, but in pixels in each direction) due to memory and other
>hardware. 
>
>        The H.264 RTP SDP model includes (in some cases) receiver
>capabilities in the fmtp (and sometimes sender capabilities).  They do
>include a buffer size in max-dpb, but not the other items we're talking
>about.
>
>  
>
>>>>Once this is clearer, we can then decide whether this is one parameter
>>>>or possibly more. So far, all variations, whether motivated by frame
>>>>buffer size or display size, seem to have the same effect: limit what
>>>>the receiver is willing to receive or indicate what the sender will
>>>>send, where the former seems somewhat more important.
>>>>        
>>>>
>
>  
>
>>>I think one needs to make clear separation between frame-rate and
>>>frame-size. However I do not think that this is sufficient at all. There
>>>are so many other parameters in relation to decoding capabilities that
>>>are to be considered. A few of these are: number of reference frames,
>>>bit-rate according to the buffer model, number of macro blocks possible
>>>to decode in a certain time interval, codec features.
>>>      
>>>
>
>  
>
>>I'm trying to find a common, useful subset that has high impact on either
>>interoperability or efficiency. Clearly, there will always be
>>codec-specific parameters, such as the ones you mention. Maximum display
>>size and maximum frame buffer size, however, seem pretty universal.
>>    
>>
>
>Display refresh speed is a pretty universal one too - it tells you updates
>faster than that probably aren't useful (even if decodable), and also
>probably tells you that this is a preferred speed.  I'll admit this one
>might require a little thought compared to sizes, but it is important.  If
>I have an LCD that updates at most 20 times per second, 60 frame per second
>video is a pretty big waste of bandwidth.
>
>  
>
_______________________________________________
mmusic mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mmusic

---

From: Randell Jesup <[email-masked]>
Date: 22 Mar 2005 15:44:45 -0500

>From the draft:
   - The "a=3gpp-videopostdecbufsize" attribute is used to indicate
      the size of the H.264 video post-decoder buffer.

Why is this limited to H.264?  Or should I ask, is this merely cataloging
the usage 3GPP puts these new attributes to, or are they expected to be
of use outside of 3GPP?

  3.2. Video Frame Size attribute

   This media-level attribute provides the receiver with the largest
   picture size a specific H.263 payload type will carry within the
   session. The attribute has the following form (see 5.3.3.2 of [1]):

This doesn't specify what directionality is being referred to; therefore I
assume that "receiver" means the device receiving the SDP, and this
attribute means that the framesize given is the largest that the sender is
willing to receive for that payload type.  Or is this bidirectional?  Or
is the size it will send for that payload?  I expect the first (size
willing to receive).

Also, this is limited to H.263 apparently.  (Why?  I assume because in 3GPP
it's needed for H.263 and not for other H.264, which takes the sizes from
the profile.)

Are a=alt and some of these others intended to be used outside of 3GPP?  It
appears they could be an alternative to the other grouping draft/RFC.  It's
unfortunate that they can't operate on a media line (profiles, etc), but
it's not surprising.  They are obviously rather targeted at 3GPP's needs.

There also doesn't seem to be a direct way to specify alternatives A+B or
A+C or B+D or just E or just A (for example).  You can't say that a valid
grouping has one of alternatives for one stream, and no alternative (no
stream, reject the media line) for the other.  You can probably cheat it by
specifying some sort of no-op or nil-bandwidth codec as one of the
alternatives.

Which brings me back to: all of these are defined in the context of 3GPP.
Are they intended or expected to be used more widely?  Or will we get yet
another set of attributes that are mostly like (but not the same) as sets
used elsewhere?  Are we going to define widely-usable solutions to these
issues?

>draft-ietf-avt-rfc2429-bis-04.txt

Again, that applies to H.263, not a general SDP attribute (currently
at least).

-- 
Randell Jesup, Worldgate Communications, ex-Scala, ex-Amiga OS team
[email-masked]

---

From: Aron Rosenberg <[email-masked]>
Date: Tue, 22 Mar 2005 13:05:37 -0800

Comments Inline -

>the usage 3GPP puts these new attributes to, or are they expected to be
Not sure, but we use the field in systems other than H.264 to indicate 
the same thing. I would be in favor of making a generic field name not 
limited to 3GPP or having the 3GPP in it - We would have to work out the 
overlap with Magnus on the current 3GPP draft?

>is the size it will send for that payload?  I expect the first (size
We use this unidirectional per offer / answer.

>it's needed for H.263 and not for other H.264, which takes the sizes from
Technically that is correct - Since we have some custom video codecs in 
our application we use the SDP attribute in general and the RTP profile 
in particular if it supports it. Also, not every video related media 
stream that needs the information is sent over RTP so the profile isn't 
always the best place for it.

>specifying some sort of no-op or nil-bandwidth codec as one of the
This is where it gets very interesting. As far as I can tell, you can't 
mix the methodologies. SightSpeed has taken a vendor enhanced approach 
to this right now to make sure that SDP works with offer/answer and 
multiple media types, and alt.

It may be at the point where only SDPnew can resolve it since there are 
so many existing requirements with the grouping framework, that adding 
in alt or offer lines for video is not feasible.

>used elsewhere?  Are we going to define widely-usable solutions to these
My guess is that you will see them in use for 3GPP at first and trickle 
in if no other support is provided. At moment they are useless in terms 
of interoperability.

---

From: Henning Schulzrinne <[email-masked]>
Date: Tue, 22 Mar 2005 19:51:42 -0500

Very helpful. Are these descriptions scoped so that they can be used 
outside of their respective payload or protocol (RTSP) context? It seems 
suboptimal that they are hidden in some random payload and 3GPP documents.

Aron Rosenberg wrote:
> ------------------------------------------------------------------------
>

---

From: Aron Rosenberg <[email-masked]>
Date: Tue, 22 Mar 2005 17:14:04 -0800

At the moment they are scoped only within the documents. The videosize 
is limited to 3GPP, the MPI/framerate is limited to H.263 payloads, 
etc.  I have been collecting attributes and ideas to put into a general 
SDP/video draft since it appears one is needed to bring together all the 
various changes.

So far the needs seem to be for:

FrameSizes: Max, min, or defined set - 120x96->352x288  or 
120x96,160x120,176x144,352x288
SendFrameRates: Max, min or defined set - 1->30, 1,5,10,15,20,30

(The framerate is really only useful if you want to limit the upper end 
for display reasons)

Nice to have video attributes:
PreferedColorSpace: Many video codecs can encode in different color 
spaces such as YUV or RGB. For some there is only one. Something like a 
FOURCC code. This would indicate a recommended following

Depending on how the offer/answer is used, you don't need a receive size 
since the receiver can return the values that they want to use so long 
as the sender lists all values acceptable to them. If the grouping draft 
disables the proper use of offer/answer, you can specify a RecvFrameSize 
as well.

You don't really need a separate compression/buffer and display size 
since that should be handled by the application.

Aron Rosenberg
SightSpeed - Video Phone Service that Works
<http://www.sightspeed.com>

Henning Schulzrinne wrote:

> outside of their respective payload or protocol (RTSP) context? It 
> seems suboptimal that they are hidden in some random payload and 3GPP 
> documents.
>
> Aron Rosenberg wrote:
>
>> The 3GPP ID already defines most of the attributes that you are 
>> looking for: draft-westerlund-mmusic-3gpp-sdp-rtsp-00.txt
>>
>> SightSpeed uses these to support proper Video identification and 
>> sizes over a broad range of devices. When combined with the 
>> offer/answer model, the receiver can modify the attributes to 
>> indicate lesser support of a given attribute. So long as the sender 
>> checks the answer, you can negotiate a proper media session. Most 
>> everything that you are looking for is already in some sort of 
>> proposed form. There is no need for separate Display rates or codec 
>> rates, the end device determines its lowest common rate and highest 
>> rate and offer/answers that.
>>
>> Aron Rosenberg

---

From: "Andreas J. Kassler" <[email-masked]>
Date: Wed, 23 Mar 2005 09:19:49 +0100

Hi,
We have proposed some extension/reformulation for the SDPng draft to be able
to specify requirements on 
- framesize
- framerate
and some other video and audio related parameters. You can specify if those
are expressed as a defined set,...
It also allows to specify constraints for the display like resolution, color
bit depth, and constraints on the data rate.
The nice feature is that it is also aligned with MPEG-21 DIA tools. 
The draft name is draft-guenkova-mmusic-mpeg21-sdpng-00 and was presented at
the last IETF. If you look in the annex you find all the XML elements for
expressing those.
Kind regards, Andreas Kassler

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Aron Rosenberg
Sent: den 23 mars 2005 02:14
To: Henning Schulzrinne
Cc: [email-masked]
Subject: Re: [MMUSIC] Re: Maximum non-scaled image size

Aron Rosenberg

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 23 Mar 2005 11:54:28 +0100

Hi,

Lets try to answer your questions.

First all the extensions in the draft 
(draft-westerlund-mmusic-3gpp-sdp-rtsp-00.txt) is defined for the PSS 
service and which means RTSP controlled streaming. Therefore no 
offer/answer policy is defined for any of the attributes.

> the usage 3GPP puts these new attributes to, or are they expected to be

The reason is why this is only defined for H.264, is that it wasn't 
defined and considered useful for H.263 and MPEG-4 visual when Annex G 
procedures where defined in release 5. So when H.264 was added in 
release 6, the parameter was deemed necessary and therefore introduced 
for that codec.

> is the size it will send for that payload?  I expect the first (size

In PSS there can't be anything else than being a stream property. The 
behavior in offer/answer is undefined.

> it's needed for H.263 and not for other H.264, which takes the sizes from

Correct, the image size is defined by the MPEG-4 config string, and the 
H.264 sequence parameter set. Therefore it is not needed for these 
codecs. The intention is to avoid duplicating information that is 
present to avoid risk of inconsistency.

> it's not surprising.  They are obviously rather targeted at 3GPP's needs.

They where only targeted for 3GPP. The reason doesn't operate on media 
lines is that it would generate difficult problems of interpretation and 
  validity.

> specifying some sort of no-op or nil-bandwidth codec as one of the

The reason why these limitations do not exist, is that there is no easy 
way to negotiate this in streaming. Therefore the whole alternative 
mechanism is based on the assumption that a receiver will need to be 
capable of handling any of the alternatives that the server can send. 
PSS relies on capability exchange to avoid sending media configurations 
that are not supported by the client. Therefore a receiver is only given 
the choice to select a bit-rate or language alternative that suits it 
from a list that otherwise will be supported.

If your interested in doing a real negotiation along the lines above, 
you definitely need another mechanism. In my opinion this type of 
negotiation needs a new and better model for negotiation and 
description. SDP is not your answer.

> Which brings me back to: all of these are defined in the context of 3GPP.

Yes, and a specific application, namely streaming with a constrained 
dynamic range.

> Are they intended or expected to be used more widely?  

No!

Or will we get yet
> used elsewhere?  Are we going to define widely-usable solutions to these

Good question.

Cheers

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

---

From: Colin Perkins <[email-masked]>
Date: Wed, 23 Mar 2005 13:41:32 +0000

There are also similar parameters defined in  
draft-ietf-avt-uncomp-video-06.txt for that format.

Colin

On 23 Mar 2005, at 01:14, Aron Rosenberg wrote:

> etc.  I have been collecting attributes and ideas to put into a  
> general SDP/video draft since it appears one is needed to bring  
> together all the various changes.
>
> (The framerate is really only useful if you want to limit the upper  
> end for display reasons)
>
>
> spaces such as YUV or RGB. For some there is only one. Something like  
> a FOURCC code. This would indicate a recommended following
>
> Depending on how the offer/answer is used, you don't need a receive  
> size since the receiver can return the values that they want to use so  
> long as the sender lists all values acceptable to them. If the  
> grouping draft disables the proper use of offer/answer, you can  
> specify a RecvFrameSize as well.
>

---

From: Henning Schulzrinne <[email-masked]>
Date: Wed, 23 Mar 2005 09:16:03 -0500

Seems like an ever increasing mess. Since some of these are still 
drafts, maybe this is a good (last) opportunity to unify these. Clearly, 
the need has been arrived at by numerous independent parties...

---

From: "Christian Timmerer \(ITEC\)" <[email-masked]>
Date: Wed, 23 Mar 2005 15:25:56 +0100

Dear Magnus, Randell,
  let me jump into this discussion.

> > used elsewhere?  Are we going to define widely-usable solutions to these
[Christian Timmerer (ITEC)] Indeed a good question. We're quite interested in this area and we would like to contribute, especially
we investigated some of the "sets used elsewhere". In particular, we tried to incorporate usage environment descriptions as
specified in the MPEG-21 Digital Item Adaptation standard (i.e., providing information about the terminal capabilities and network
characteristics among others) into SDPng.

What do you think?

Thanks.
Best regards,
 -Christian