Subject: Re: [MMUSIC] draft-johansson-mmusic-image-attributes

From: "Ingemar Johansson S" <[email-masked]>
Date: Wed, 2 Jul 2008 12:13:58 +0200

Hi

As a follow up..

Had a closer look at sprop-parameter-sets and its relation to imageattr. To my knowledge it is only a few of the elements in sprop-parameter-sets that may conflict with imageattr.
 
These are:
pic_width_in_mbs_minus1
pic_height_in_map_units_minus1
frame_cropping...

I may miss some elements but my point is that if one simply ignore sprop-parametr-sets completely one will miss other non-imageattr related elements that may be useful.
The question of "selective ignorance" or sprop-parameter-sets may be an implementation issue, I can perfectly well understand if something like is is not possible with an off-the-shelf H.264 decoder but still I don't belive that a general "ignore sprop-parameter-sets" is the best alternative

Another thing that has relevance is that if a 2nd offer/answer round is done (needed to support asymmetric setup), then the sprop-parameter-sets in the 2nd offer/answer can be tuned to match the negotiated imagesize, this makes decoder setup easier. This is of course an H.264 specific feature.

Regarding the already existing possibilities to specify image size in H.264 and other codec payload formats:
I beleive here one must make a distinction between what can be specified and what can be negotiated. H.264 makes it possible for the offerer specify what is transmitted, in other words it is not negotiation.
H.263 makes it possible to use a CUSTOM size and this may work to some extent but in my humble opinion this has relatively limited potential to do actual imagesize negotiation as is proposed in the imageattr draft.

Regards
Ingemar

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Ingemar Johansson S
> Sent: den 30 juni 2008 14:02
> To: Even, Roni; Colin Perkins
> Cc: [email-masked]
> Subject: Re: [MMUSIC] draft-johansson-mmusic-image-attributes
> 
> Hi
> 
> Thanks for comments, answers inline below
> 
> /Ingemar 
> 
> > -----Original Message-----
> > From: Even, Roni [mailto:[email-masked]]
> > Sent: den 30 juni 2008 08:45
> > To: Colin Perkins; Ingemar Johansson S
> > Cc: [email-masked]
> > Subject: RE: [MMUSIC] draft-johansson-mmusic-image-attributes
> > 
> > Hi,
> > I think Colin makes good point and would like to add that 
> bullet 1 in 
> > section 5 may cause a interoperability problem if the answerer 
> > supports only the sprop parameter. My suggestion is to 
> specify that if 
> > imageattr is supported that the sprop parameter will be ignored.
> OK, will check this out
>  
> > I also want to add that image size are also specified in 
> other payload 
> > formats and not only in H.264.
> I find it specified for H.263 (CUSTOM=x,y,MPI). One issue I 
> see is that it, in normal cases, specifies what the offerer 
> wants to receive. Suppose that an fmtp line is as follows.
>    CUSTOM=360,240,2;CIF=4;QCIF=3;SQCIF=2
> This line would indicate that the custom size (360x240) is 
> the most preferred in order to avoid rescaling. If the 
> answerer accepts this, then we are home free. However if it 
> is not supported the next step is CIF (352x288), if we 
> suppose that this is OK then the receiver of the media will 
> need to do rescaling to fit the image within the desired 
> 360x240 area as the implementor wants to fit other stuff such 
> as menus, buttons and decorative frames etc on the display 
> area aswell.
> I would see the options listed on the fmtp line as 
> restrictions related to decoder(+encoder) complexity while 
> imageattr is a way to select desired imagesize on the screen. 
> From that perspective it may well be that the "fr=" parameter 
> should not be present for imageattr.
>  
> > The justification for image size is for supporting screen 
> size and low 
> > compute and that is why the image attribute by itself is 
> not giving a 
> > full answer. Low compute can be achieved by specifying the relation 
> > between frame rate and size like in H.263 and H.261 payload 
> > specifications or by having a max macro blocks per second 
> associated 
> > like in H.264 which applies that same limitation as the 
> frame rate per 
> > resolution in H.263/H.261.
> > By specifying the maximum image size and the maximum Macro 
> Blocks per 
> > second you can allow slower frame rate at the maximum 
> resolution and 
> > higher frame rate at lower resolution.
> I believe I agree here. There is probably little reason for 
> imageattr to interfere with this. I made the "fr=" parameter 
> optional in the draft but it is possible that it should not 
> even be allowed as it may cause too much ambiguities.
> 
>  
> > Roni Even
> > 
> > 
> > 
> > > -----Original Message-----
> > > From: [email-masked] [mailto:[email-masked]] On 
> > > Behalf Of Colin Perkins
> > > Sent: Sunday, June 29, 2008 10:00 PM
> > > To: Ingemar Johansson S
> > > Cc: [email-masked]
> > > Subject: Re: [MMUSIC] draft-johansson-mmusic-image-attributes
> > > 
> > > Hi,
> > > 
> > > The draft lists several issues in section 5. There are also
> > possible
> > > feature interactions with the "a=framerate:", "a=quality:" and 
> > > "a=orient:" attributes in RFC 4566. And, RFC 4175 also
> > defines width
> > > and height parameters which may conflict. I doubt any of 
> these are 
> > > major issues, but the interactions need to be specified.
> > > 
> > > Cheers,
> > > Colin
> > > 
> > > 
> > > 
> > > On 26 Jun 2008, at 15:52, Ingemar Johansson S wrote:
> > > > Hi
> > > >
> > > > The image attribute draft has been updated to version -01
> > with some
> > > > major changes.
> > > > + Support for Sample Aspect Ratios (SAR) Offer/answer for
> > asymmetric
> > > > + display capabilities Picture aspect Ratio range to limit the 
> > > > + number of x and y
> > > > resolution options
> > > >
> > > >
> > > > Comments are welcome
> > > > /Ingemar
> > > >
> > > > ======================================
> > > > Filename:	 draft-johansson-mmusic-image-attributes
> > > > Revision:	 01
> > > > Title:		 Negotiation of Generic Image Attributes in SDP
> > > > Creation_date:	 2008-06-26
> > > > WG ID:		 Independent Submission
> > > > Number_of_pages: 13
> > > >
> > > > Abstract:
> > > > This draft proposes a new generic session setup attribute
> > to make it
> > > > possible to negotiate different image attributes such as
> > image size.
> > > > A possible use case is to make it possible for a e.g a low-end
> > > > hand- held terminal to display video without the need to
> > rescale the
> > > > image, something that may consume large amounts of memory and 
> > > > processing power.  The draft also helps to maintain an optimal 
> > > > bitrate for video as only the image size that is desired by the 
> > > > receiver is transmitted.
> > > > http://tools.ietf.org/id/draft-johansson-mmusic-image-
> > > > attributes-01.txt
> > > >
> > > > ======================================
> > > > *******************************************
> > > > Ingemar Johansson
> > > > Senior Research Engineer, IETF "nethead"
> > > > EAB/TVP - Multimedia Technologies
> > > > Ericsson Research Ericsson AB
> > > > Box 920 S-971 28 Luleå, Sweden
> > > > Tel: +46 (0)8 4043042
> > > > ECN: 850-43042
> > > > ECC: 850-43074
> > > > Mobile: +46 (0)730 783289
> > > > *******************************************
> > > > _______________________________________________
> > > > mmusic mailing list
> > > > [email-masked]
> > > > https://www.ietf.org/mailman/listinfo/mmusic
> > > 
> > > 
> > > 
> > > --
> > > Colin Perkins
> > > http://csperkins.org/
> > > 
> > > 
> > > 
> > > _______________________________________________
> > > mmusic mailing list
> > > [email-masked]
> > > https://www.ietf.org/mailman/listinfo/mmusic
> > 
> _______________________________________________
> mmusic mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmusic
> 
_______________________________________________
mmusic mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/mmusic

---

From: "Even, Roni" <[email-masked]>
Date: Wed, 2 Jul 2008 13:37:22 +0300

Ingemar,

I meant selective ignore that needs to be specified, otherwise you may encounter interoperability problems unless you deprecate the sprop and other image size parameters in H.264.

You should look at sar parameter in draft-ietf-avt-rtp-h264-params-01 and the max mbps, and fs in RFC 3984. All this parameters have to do with image size that the receiver wants to receive or that the sender can send based on the direction of the stream. They are used for negotiation..

You should also take into consideration the issue of the level parameter which for H.264 limits the maximum image size and also in H.263-2000 if you use the level parameter.
The custom parameter in H.263 systems is used today by video conferencing system for receiver to request a specific size.

BTW: the new H.241 parameters will also be submitted to AVT as new H.264 parameters.
 
Roni

> From: Ingemar Johansson S [mailto:[email-masked]]
> Sent: Wednesday, July 02, 2008 1:14 PM
> To: Ingemar Johansson S; Even, Roni; Colin Perkins
> Cc: [email-masked]
> Had a closer look at sprop-parameter-sets and its relation to
> imageattr. To my knowledge it is only a few of the elements in sprop-
> parameter-sets that may conflict with imageattr.
> 
> These are:
> pic_width_in_mbs_minus1
> I may miss some elements but my point is that if one simply ignore
> sprop-parametr-sets completely one will miss other non-imageattr
> related elements that may be useful.
> The question of "selective ignorance" or sprop-parameter-sets may be an
> implementation issue, I can perfectly well understand if something like
> is is not possible with an off-the-shelf H.264 decoder but still I
> don't belive that a general "ignore sprop-parameter-sets" is the best
> alternative
> 
> Another thing that has relevance is that if a 2nd offer/answer round is
> done (needed to support asymmetric setup), then the sprop-parameter-
> sets in the 2nd offer/answer can be tuned to match the negotiated
> imagesize, this makes decoder setup easier. This is of course an H.264
> specific feature.
> 
> I beleive here one must make a distinction between what can be
> specified and what can be negotiated. H.264 makes it possible for the
> offerer specify what is transmitted, in other words it is not
> negotiation.
> H.263 makes it possible to use a CUSTOM size and this may work to some
> extent but in my humble opinion this has relatively limited potential
> to do actual imagesize negotiation as is proposed in the imageattr
> draft.
> 
> Regards
> Ingemar
>

---

From: "Ingemar Johansson S" <[email-masked]>
Date: Wed, 2 Jul 2008 13:45:15 +0200

Hi

Answer inline below. I will not update the draft until after the Dublin meeting as there may pop up other  considerations. I believe a major task is to formulate something that integrates well with existing and upcoming RFC's.

/Ingemar 

> Sent: den 2 juli 2008 12:37
> To: Ingemar Johansson S; Colin Perkins
> Cc: [email-masked]
> I meant selective ignore that needs to be specified, 
> otherwise you may encounter interoperability problems unless 
> you deprecate the sprop and other image size parameters in H.264.
OK, thanks
 
> You should look at sar parameter in 
> draft-ietf-avt-rtp-h264-params-01 and the max mbps, and fs in 
> RFC 3984. All this parameters have to do with image size that 
> the receiver wants to receive or that the sender can send 
> based on the direction of the stream. They are used for negotiation..
Had a look at this draft and the SAR negotiation here is similar to what is specified in imageattr, not exactly the same though. I have not yet considered max mbps and fs in these drafts.

> You should also take into consideration the issue of the 
> level parameter which for H.264 limits the maximum image size 
> and also in H.263-2000 if you use the level parameter.
> The custom parameter in H.263 systems is used today by video 
> conferencing system for receiver to request a specific size.
OK
 
> > The question of "selective ignorance" or 
> sprop-parameter-sets may be 
> > an implementation issue, I can perfectly well understand if 
> something 
> > like is is not possible with an off-the-shelf H.264 decoder 
> but still 
> > I don't belive that a general "ignore sprop-parameter-sets" is the 
> > best alternative
> > 
> > Another thing that has relevance is that if a 2nd 
> offer/answer round 
> > is done (needed to support asymmetric setup), then the 
> > sprop-parameter- sets in the 2nd offer/answer can be tuned to match 
> > the negotiated imagesize, this makes decoder setup easier. 
> This is of 
> > course an H.264 specific feature.
> > 
> > Regarding the already existing possibilities to specify 
> image size in
> > H.264 and other codec payload formats:
> > > I find it specified for H.263 (CUSTOM=x,y,MPI). One issue 
> I see is 
> > > that it, in normal cases, specifies what the offerer wants to 
> > > receive. Suppose that an fmtp line is as follows.
> > > This line would indicate that the custom size (360x240) 
> is the most 
> > > preferred in order to avoid rescaling. If the answerer 
> accepts this, 
> > > then we are home free. However if it is not supported the 
> next step 
> > > is CIF (352x288), if we suppose that this is OK then the 
> receiver of 
> > > the media will need to do rescaling to fit the image within the 
> > > desired 360x240 area as the implementor wants to fit other stuff 
> > > such as menus, buttons and decorative frames etc on the 
> display area 
> > > aswell.
> > > I would see the options listed on the fmtp line as restrictions 
> > > related to decoder(+encoder) complexity while imageattr 
> is a way to 
> > > select desired imagesize on the screen.
> > > From that perspective it may well be that the "fr=" 
> parameter should 
> > > not be present for imageattr.
> > >
> > > > full answer. Low compute can be achieved by specifying the 
> > > > relation between frame rate and size like in H.263 and H.261 
> > > > payload specifications or by having a max macro blocks 
> per second
> > > associated
> > > optional in the draft but it is possible that it should 
> not even be 
> > > allowed as it may cause too much ambiguities.
> > >
> > >
> > > > Roni Even
> > > >
> > > >
> > > >
> [mailto:[email-masked]] 
> > > > > On Behalf Of Colin Perkins
> > > > > > bitrate for video as only the image size that is desired by 
> > > > > > the receiver is transmitted.
> > > > > > EAB/TVP - Multimedia Technologies Ericsson Research 
> Ericsson 
> > > > > > AB Box 920 S-971 28 Luleå, Sweden
> > > > > > Tel: +46 (0)8 4043042
> > > > > > ECN: 850-43042
> > > > > > ECC: 850-43074

---

From: "Jean-Francois Mule" <[email-masked]>
Date: Thu, 3 Jul 2008 11:03:13 -0600

Hi Ingemar,

   It would be good for this draft to include a few requirements that
you think the solution must meet.
Reading through the draft, the comments posted by Roni on the 3GPP
liaison that relate to your draft, and other comments from wg
participants, it seems that a few requirements could help provide more
context and clear some of the questions asked, especially w.r.t. the
interactions between this new SDP attribute, existing SDP attributes and
the various codec RTP parameters that might be close to the parameter as
pointed by Roni and Colin.

   For your consideration, the following could be candidate topics to
address in a requirement section:
- specify precisely what the offerer must be able to indicate with such
an image attribute: size, dimensions, resolution, aspect ratio of image
a physical device or applications running the SDP agent is able to
receive/display/render (with or without rescaling?) 
- indicate the SDP level of the attribute
- have an explicit requirement that the solution must interwork with
codecs that have advanced parameters for negotiating similar parameters

On a different note about use cases for this, what about multicast?
Could this type of parameter be useful in session announcements to
provide a hint about the resolution of the media in a particular media
block?

Jean-Francois.

---

From: "Ingemar Johansson S" <[email-masked]>
Date: Fri, 4 Jul 2008 15:22:32 +0200

Hi

comments inline.

Ingemar

> From: Jean-Francois Mule [mailto:[email-masked]] 
> Sent: den 3 juli 2008 19:03
> To: Ingemar Johansson S
> Cc: [email-masked]; Even, Roni; Colin Perkins
>    It would be good for this draft to include a few 
> requirements that you think the solution must meet.
> Reading through the draft, the comments posted by Roni on the 
> 3GPP liaison that relate to your draft, and other comments 
> from wg participants, it seems that a few requirements could 
> help provide more context and clear some of the questions 
> asked, especially w.r.t. the interactions between this new 
> SDP attribute, existing SDP attributes and the various codec 
> RTP parameters that might be close to the parameter as 
I too believe that it would be a good idea to write down a requirement
spec in a future version of the draft

> 
>    For your consideration, the following could be candidate 
> topics to address in a requirement section:
> - specify precisely what the offerer must be able to indicate 
> with such an image attribute: size, dimensions, resolution, 
> aspect ratio of image a physical device or applications 
> running the SDP agent is able to receive/display/render (with 
> or without rescaling?)
> - have an explicit requirement that the solution must 
> interwork with codecs that have advanced parameters for 
> negotiating similar parameters
> 
> Could this type of parameter be useful in session 
> announcements to provide a hint about the resolution of the 
> media in a particular media block?
I have tried to "unthink multicast" for he moment and stick to
point-to-point scenarios or conferences where negotiaton is done between
the focus and the paticipants. General multicast is yet a level of
complexity.