Subject: Re: [AVT] [MMUSIC] spatial-resolution parameter for RFC 3984bis

From: Kyunghun Jung <[email-masked]>
Date: Tue, 29 Jul 2008 23:53:48 +0000 (GMT)

Thank you Mr. Even Roni for the information.

We would like to ask experts in MMUSIC and AVT to pay a special interest
to the relationship

between imageattr and config/sprop-parameter-set. How to stack the set of
information in the SDP offer

will influence the session setup.

For example, suppose imageattr specifies image sizes over multiple levels.
The following form might be necessary.

m=video 49154 RTP/AVPF 99

b=AS:48

b=RS:0

b=RR:2500

a=rtpmap:99 MP4V-ES/90000

a=imageattr:99 [x=x1,y=y1] [x=x2,y=y2] [x=x3,y=y3]

a=fmtp:99 profile-level-id=a; \
config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1

a=fmtp:99 profile-level-id=b; \

config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_2

a=fmtp:99 profile-level-id=c; \

config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_3

In case imageattr specifies image sizes within a single level.

a=imageattr:99 [x=x1,y=y1] [x=x2,y=y2] [x=x3,y=y3]

a=fmtp:99 profile-level-id=8

a=fmtp:99 config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1

a=fmtp:99 config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_2

a=fmtp:99 config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_3

The above forms are only basic possibilities and more systematic design and
suggestion are

requested from MMUSIC and AVT experts.

Have a nice meeting at Dublin!

Kyunghun Jung

------- Original Message -------
Sender : Even, Roni<[email-masked]>
Date : 2008-07-30 00:13 (GMT+09:00)
Title : RE: [MMUSIC] [AVT] spatial-resolution parameter for RFC 3984bis

Hi,

First of all, ITU-T documents in pdf format are free at

http://www.itu.int/rec/T-REC/e

As for custom format inH.263. RFC 4629 section 8.1.1 had the custom

resolution, no relation to specific level.

CUSTOM: Specifies the MPI (Minimum Picture Interval) for a

custom-defined resolution.  The custom parameter receives three

comma-separated values, Xmax, Ymax, and MPI.  The Xmax and Ymax

parameters describe the number of pixels in the X and Y axis and

must be evenly divisible by 4.  The permissible values for MPI are

integer values from 1 to 32, which correspond to a maximum frame

rate of 30/(1.001 *the specified value).

Also note the next sentence

A system that declares support of a specific MPI for one of the

resolutions SHALL also implicitly support a lower resolution with

the same MPI.

Roni Even

> -----Original Message-----

> From: Randell Jesup [mailto:[email-masked]]

> Sent: Tuesday, July 29, 2008 5:08 PM

> To: [email-masked]

> Cc: Ingemar Johansson S; [email-masked]; Even, Roni; Ye-

> [email-masked]; [email-masked]

> Subject: Re: [MMUSIC] [AVT] spatial-resolution parameter for RFC

> 3984bis

> Kyunghun Jung <[email-masked]> writes:

> >Some clarifications are as follows.

> >

> >

> >

> >>So far as I know (and I didn't recheck), H.263 only allows for

> resolutions

> >>specified in the fmtp (though QCIF and CIF are I think required).

If

> you

> >>want 344x288, you need to add a CUSTOM resolution to the fmtp line.

> And

> >>the mpi must be an integer in the fmtp.  Now, if both sides use

this,

> then

> >>those restrictions could be relaxed.

> >

> >In H.263, Custom Picture Format (CPFMT) is supported at level 50 or

> >higher.

> >

> >3GPP requires only level 45 of H.263 for its mobile video telephony

> and

> >whether its level is increased further is doubtful. In other words,

> MPEG-4

> >and H.264 are expected to utilize imageattr a lot.

> I think you misunderstand my comment.  I was referring the the SDP

fmtp

> line for H.263, where custom resolutions (other then CIF, etc) can be

> specified.

> Since the ITU specs are a pain to come by (I think I have an old H.263

> and

> Annexes stashed somewhere, but it isn't handy at the moment, from

> home), I

> can't easily look up the Annex X profile/level stuff; I just know how

> the

> SDP works from the IETF spec.  (I tried to look it up online - Ugh,

> requires TIES account.  The ITU appears to not want anyone except

large

> companies to implement their specs.)  :-(

> It sounds like "CUSTOM" isn't of any use for many scenarios.

> In any case, I was responding to Ingemar's example that used custom

> resolutions in H.263.

> >Sincerely yours,

> >

> >Kyunghun Jung

> >

> >

> >

> >------- Original Message -------

> >Sender : Randell Jesup<[email-masked]>

> >Date : 2008-07-29 00:49 (GMT+09:00)

> >Title : Re: [MMUSIC] [AVT] spatial-resolution parameter for RFC

> 3984bis

> >

> >"Ingemar Johansson S" <[email-masked]> writes:

> >>To answer Roni's and Ye-Kui's comments I believe that the image

> attribute

> >>can be used for 3. However, looking in perspective I am not longer

> sure

> >>that the possibility to specify a framerate range makes sense as it

> does

> >>not connect  too wellto reality when it comes to e.g H.264 which

> talks

> >>about macroblocks per second. The conclusion is then that the

> framerate

> >>(or MBs) is determined from the fmtp line.

> >>This means that if the imageattr specifies a large offered range the

> frame

> >>rate is limited by what is possible for the given preferred

> resolution.

> >>

> >>As an example, consider H.263 with the fmtp line

> >> CIF=4;QCIF=3;SQCIF=2;

> >>the imageattr may then be specified as

> >>a=imageattr:97 [x=[128:8:352],y=[96:8:288],par=[1.2,1.3]]

> >>If the receiver prefers the resolution 344x288 (slightly less than

> CIF) the maximum frame rate would be 30/(1.0001*4) ~ 7.5Hz.

> >>If the prefered resolution is 160x128 (slightly less than QCIF) then

> the max frame rate is 30/(1.0001*3) Hz and so on.

> >

> >So far as I know (and I didn't recheck), H.263 only allows for

> resolutions

> >specified in the fmtp (though QCIF and CIF are I think required).  If

> you

> >want 344x288, you need to add a CUSTOM resolution to the fmtp line.

> And

> >the mpi must be an integer in the fmtp.  Now, if both sides use this,

> then

> >those restrictions could be relaxed.

> >

> >>For the H.264 case the frame rate depending of the preferred image

> size

> >>would be limited according to the maximum framerate repending on the

> max

> >>MBs and the resolution given in table A-6 in the H.264 spec. For

> instance

> >>for level 2 an image size of 344x280 which is less than CIF but

> larger

> >>than QVGA would yield a max frame rate of 30Hz.

> >

> >My suggestion would be for this to express preferences within the

> normal

> >negotiated resolution and framerate space.  i.e for 264 the level and

> >max-fs and max-mbps would determine the *maximum* frame and framerate

> >supported, but this would be used to decide what to actually send.

> >

> >See my original email below -- look for:

> > "I would suggest instead (and did in the thread back in March)"

> >

> >>

> >>Would this distincion between framerate (or MBs) and preferred image

> size work?

> >>

> >>/Ingemar

> >>

> >>

> >>

> >>> -----Original Message-----

> >>> From: Even, Roni [mailto:[email-masked]]

> >>> Sent: den 28 juli 2008 11:32

> >>> To: [email-masked]; Ingemar Johansson S; [email-masked]

> >>> Cc: [email-masked]

> >>> Subject: RE: [AVT] spatial-resolution parameter for RFC 3984bis

> >>>

> >>> Hi,

> >>> I think that we have to be clear about the usage of the parameter

> >>>

> >>> 1. Announce sending of the specific resolution - just

> >>> resolution 2. Request a specific resolution - Receiver wants

> >>> a specific resolution 3. Negotiate available resolutions -

> >>> The offer says, I can do CIF, QCIF, 1080P,... here it may

> >>> dependent in H.263 and H.261 on the frame rate while in H.264

> >>> the limit is in the macroblocks per second which will specify

> >>> the maximum frame rate. The reason for this offer is because

> >>> it offers the operation points that the offerer can have. The

> >>> level parameter in H.263 and H.264 gives a maximum but does

> >>> not imply support for any resolution that the answer would

> >>> like to receive that falls within the range of the level.

> >>>

> >>> I think the img attribute address 1 and 2, I am not sure about 3

> >>>

> >>> Roni Even

> >>>

> >>> > -----Original Message-----

> >>> > From: [email-masked] [mailto:[email-masked]]

> >>> > Sent: Monday, July 28, 2008 1:11 PM

> >>> > To: [email-masked]; [email-masked]

> >>> > Cc: Even, Roni; [email-masked]

> >>> > Subject: RE: [AVT] spatial-resolution parameter for RFC 3984bis

> >>> >

> >>> >

> >>> > Interesting! I just wrote the following email, planned to

> >>> send to the

> >>> > MMUSIC mailing list. Now this comes only to AVT. Feel free

> >>> to forward

> >>> > it to MMUSIC (don't know whether it is allowed to cross posting)

> or

> >>> > let me to send that, if needed.

> >>> >

> >>> >

> >>> > After a reading of this draft (draft-johansson-mmusic-image-

> >>> > attributes), I think the sprop-spatial-resolution parameter in

> the

> >>> > RFC3984bis can go. It is completely covered by imageattr.

> >>> >

> >>> > A couple of comments to draft-johansson-mmusic-image-attributes.

> >>> >

> >>> > - It would be helpful to introduce the methods in payload

> >>> format specs

> >>> > for H.261, H.263 and H.264. In particular, for the one I am

> >>> familiar

> >>> > with (i.e. H.264), I am not sure how exactly image size

> >>> together with

> >>> > frame rate is negotiated.

> >>> >

> >>> > - Re the following paragraph, I am not sure which parameters are

> >>> > "similar features" and MUST NOT be specified in imageattr.

> >>> Basically,

> >>> > H.264 sequence parameter set (SPS)may contain all

> >>> information relating

> >>> > to what may be included in imageattr according to the curent

> draft.

> >>> > For example, the VUI part of SPS may even contain frame

> >>> rate related

> >>> > synatx elements.

> >>> >    o  Conflict with sprop-parameter-set: H.264 defines the

sprop-

> >>> >       parameter-set which may define features that are

> >>> defined in this

> >>> >       draft.  To avoid conflicts, sessions that use the

imageattr

> >>> >       attribute MUST NOT specify similar features in

> >>> sprop-parameter-

> >>> >       set.

> >>> >

> >>> > BR, YK

> >>> >

> >>> > >-----Original Message-----

> >>> > >From: ext Ingemar Johansson S

> >>> > >[mailto:[email-masked]]

> >>> > >Sent: Monday, July 28, 2008 1:04 PM

> >>> > >To: [email-masked]

> >>> > >Cc: Wang Ye-Kui (Nokia-NRC/Tampere); Even, Roni; Randell Jesup

> >>> > >Subject: Re: [AVT] spatial-resolution parameter for RFC 3984bis

> >>> > >

> >>> > >Hi

> >>> > >

> >>> > >

> >>> > >Given the discussion below I really wonder if it is not a

better

> >>> > >alternative to specifiy preferred image sizes using a dedicated

> >>> > >(generic) SDP attribute.

> >>> > >Even though this is solved in some way in 3984bis we still have

> to

> >>> > >provide with a solution for MPEG-4 or H.263 later on.

> >>> > >I drafted a proposed solution for an "a=imageattr" attribute

> some

> >>> > >time ago that may be useful

> >>> > >http://tools.ietf.org/id/draft-johansson-mmusic-image-

> attributes-

> >>> > 01.txt

> >>> > >The draft provides with a solution that may work but lacks a

> >>> > >requirement spec to outline exactly what the attribute

> >>> aims for. Also

> >>> > >extra work is needed to avoid conflicts with related

> >>> payload format

> >>> > >parameters.

> >>> > >

> >>> > >Regards

> >>> > >Ingemar

> >>> > >

> >>> > >

> >>> > >=============================

> >>> > >To: <Ye-Kui.Wang at nokia.com>

> >>> > >Subject: Re: [AVT] spatial-resolution parameter for RFC 3984bis

> >>> > >From: Randell Jesup <rjesup at wgate.com>

> >>> > >Date: Mon, 14 Jul 2008 09:35:18 -0400

> >>> > >Cc: roni.even at polycom.co.il, avt at ietf.org

> >>> > >

> >>> > ><Ye-Kui.Wang at nokia.com> writes:

> >>> > >>The resolution must be restricted by the level part of

> >>> > >>profile-level-id and max-fs. Therefore, if a configuration is

> agreed

> >>> > >>(or agreeable), then the answerer must request a resolution

> under

> >>> > >>the constraint, and the offerer must accept the request as

long

> as

> >>> > >>it meets the restriction.  This is similar as sprop-parameter-

> sets,

> >>> > >>as long as it is complaint with the agreed (or agreeable)

> >>> > >>configuration, the other side must accept it.

> >>> > >

> >>> > >I would suggest instead (and did in the thread back in March):

> that

> >>> > >as a receiver property, it is the *preferred* maximum frame

> size.

> >>> > >The receiver is required to decode and handle any size up to

the

> size

> >>> > >implied by the level (and max-fs if given), but the receiver

> prefers

> >>> > >this size (for example, it may support 1920x1080, but only have

> an

> >>> > >LCD display that can display 1200x720, so there's no *need* to

> send

> >>> > >it anything higher (and the encoder may prefer to avoid

encoding

> more

> >>> > >macroblocks, especially in an MCU that shares resources)).  In

> other

> >>> > >cases, the sender may have the stream already encoded in

> different

> >>> > >resolutions, and this can help select the best one to send to

> save

> >>> > >encoding (and bandwidth) resources.

> >>> > >

> >>> > >There is no requirement that the encoder scale, but if it wants

> to it

> >>> > >can.

> >>> > >

> >>> > >It also (if I remember) provides aspect-ratio information.

> >>> > >

> >>> > >>Would it work this way?

> >>> > >>

> >>> > >>BR, YK

> >>> > >>

> >>> > >>>-----Original Message-----

> >>> > >>>From: ext Even, Roni [mailto:roni.even at polycom.co.il]

> >>> > >>>Sent: 14 July, 2008 10:38

> >>> > >>>To: Wang Ye-Kui (Nokia-NRC/Tampere); avt at ietf.org

> >>> > >>>Subject: RE: [AVT] spatial-resolution parameter for RFC

> 3984bis

> >>> > >>>

> >>> > >>>YK,

> >>> > >>>Just to clarify,

> >>> > >>>The text on the receiver side says that this is the requested

> >>> > >>>resolution, it does not address issue of negotiation,

> >>> what will the

> >>> > >>>sender do if it cannot send this resolution, should it

> >>> reject the

> >>> > >>>stream?.

> >>> > >>>If this is for negotiation do you except to do a new

> >>> offer answer

> >>> > >>>if the receiver wants a new resolution?

> >>> > >>>

> >>> > >>>Roni

> >>> > >>>

> >>> > >>>> -----Original Message-----

> >>> > >>>> From: avt-bounces at ietf.org [mailto:avt-bounces at

> >>> ietf.org] On

> >>> > >>>> Behalf Of Ye-Kui.Wang at nokia.com

> >>> > >>>> Sent: Sunday, July 13, 2008 7:57 PM

> >>> > >>>> To: avt at ietf.org

> >>> > >>>> Subject: [AVT] spatial-resolution parameter for RFC 3984bis

> >>> > >>>>

> >>> > >>>>

> >>> > >>>> Concluded from the discussions in the email threads listed

> >>> > >below, I

> >>> > >>>> added a new parameter, spatial-resolution, to the RFC

> >>> > >3984bis draft

> >>> > >>>> (which will be submitted by tomorrow):

> >>> > >>>>

> >>> > >>>> http://www.ietf.org/mail-

> archive/web/avt/current/msg09425.html

> >>> > >>>> http://www.ietf.org/mail-

> archive/web/avt/current/msg09650.html

> >>> > >>>>

> >>> > >>>> The parameter is specified as follows (basically as Randell

> >>> > >>>> suggested in

> >>> > >>>>

> >>> http://www.ietf.org/mail-archive/web/avt/current/msg09429.html).

> >>> > >>>>

> >>> > >>>> spatial-resolution:

> >>> > >>>> This parameter MAY be used to indicate the maximum spatial

> >>> > >>>> resolution of a NAL unit stream or the preferred spatial

> >>> > >resolution

> >>> > >>>> of a receiver.

> >>> > >>>> The value is a base16 [6] (hexadecimal) representation of

> the

> >>> > width

> >>> > >>>and

> >>> > >>>> height of the spatial resolution, in pixels, separated

> >>> by a comma.

> >>> > >>>>

> >>> > >>>> Usage in SDP offer/answer:

> >>> > >>>> - When "a=sendrecv" or no direction attribute is used,

> >>> > >>>> "spatial-resolution" indicates both the maximum spatial

> >>> > >>>> resolution of the stream sent by the declaring entity and

> the

> >>> > >preferred spatial

> >>> > >>>> resolution for receiving a stream.

> >>> > >>>> - When "a=sendonly", "spatial-resolution" indicates the

> maximum

> >>> > >>>spatial

> >>> > >>>> resolution of the outgoing stream.

> >>> > >>>> - When "a=recvonly", "spatial-resolution" indicates

> >>> the preferred

> >>> > >>>> spatial resolution for receiving a stream.

> >>> > >>>>

> >>> > >>>> Comments are welcome.

> >>> > >>>>

> >>> > >>>> BR, YK

> >>> > >>>> _______________________________________________

> >>> > >>>> Audio/Video Transport Working Group avt at ietf.org

> >>> > >>>> https://www.ietf.org/mailman/listinfo/avt

> >>> > >>>

> >>> > >>_______________________________________________

> >>> > >>Audio/Video Transport Working Group

> >>> > >>avt at ietf.org

> >>> > >>https://www.ietf.org/mailman/listinfo/avt

> >>> > >

> >>> > >

> >>> > >--

> >>> > >Randell Jesup, Worldgate (developers of the Ojo

> >>> videophone), ex-Amiga

> >>> > >OS team rjesup at wgate.com "The fetters imposed on

> >>> liberty at home

> >>> > >have ever been forged out of the weapons provided for

> >>> defence against

> >>> > >real, pretended, or imaginary dangers from abroad."

> >>> > >        - James Madison, 4th US president (1751-1836)

> >>> > >

> >>> > >*******************************************

> >>> > >Ingemar Johansson

> >>> > >Senior Research Engineer, IETF "nethead"

> >>> > >EAB/TVP - Multimedia Technologies

> >>> > >Ericsson Research Ericsson AB

> >>> > >Box 920 S-971 28 Luleaa, Sweden

> >>> > >Tel: +46 (0)8 4043042

> >>> > >ECN: 850-43042

> >>> > >ECC: 850-43074

> >>> > >Mobile: +46 (0)730 783289

> >>> > >*******************************************

> >>> > >

> >>>

> --

> Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga

> OS team

> [email-masked]

> "The fetters imposed on liberty at home have ever been forged out of

> the weapons

> provided for defence against real, pretended, or imaginary dangers

from

> abroad."

>         - James Madison, 4th US president (1751-1836)

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt

---

From: "Ingemar Johansson S" <[email-masked]>
Date: Wed, 30 Jul 2008 11:23:25 +0200

Hi
 
If I understand your email correctly we will, with the constructed SDP's
below, have the potential issue that imageattr may offer combinations (x
and y resoluions) that are not allowed for a given profile level ID. The
solution I see is that the profile levels provides with an upper (maybe
also a lower?) limit.
 
For instance if we have some (really wierd) imageattr like
a=imageattr:97 [x=[128:16:1920], y=[96:16:1280],par=[1.1 1.3]]
and the profile levels only supports a resolution up to 352x288 , the
latter limitation will apply which means that the imageattr should
infact be interpreted as
a=imageattr:97 [x=[128:16:352], y=[96:16:288],par=[1.1 1.3]]
 
This is not written anywhere in the draft and I don't right now know if
this will hold in the general case, please comment.
 
/Ingemar

________________________________

From: Kyunghun Jung [mailto:[email-masked]] 
Sent: den 30 juli 2008 00:54
To: Even, Roni
Cc: Randell Jesup; Ingemar Johansson S; [email-masked];
[email-masked]; [email-masked]
Subject: Re: RE: [MMUSIC] [AVT] spatial-resolution parameter for RFC
3984bis

	 We would like to ask experts in MMUSIC and AVT to pay a special
interest to the relationship

	between imageattr and config/sprop-parameter-set. How to stack
the set of information in the SDP offer

	For example, suppose imageattr specifies image sizes over
multiple levels. The following form might be necessary.

	b=AS:48

	b=RS:0

	b=RR:2500

	The above forms are only basic possibilities and more systematic
design and suggestion are 

	Kyunghun Jung

	------- Original Message -------
	Sender : Even, Roni<[email-masked]>
	Date : 2008-07-30 00:13 (GMT+09:00)
	Title : RE: [MMUSIC] [AVT] spatial-resolution parameter for RFC
3984bis
	
	Hi, 
	
	As for custom format inH.263. RFC 4629 section 8.1.1 had the
custom 
	resolution, no relation to specific level. 
	
	CUSTOM: Specifies the MPI (Minimum Picture Interval) for a 
	      custom-defined resolution.  The custom parameter receives
three 
	      comma-separated values, Xmax, Ymax, and MPI.  The Xmax and
Ymax 
	      parameters describe the number of pixels in the X and Y
axis and 
	      must be evenly divisible by 4.  The permissible values for
MPI are 
	      integer values from 1 to 32, which correspond to a maximum
frame 
	      rate of 30/(1.001 *the specified value). 

	      A system that declares support of a specific MPI for one
of the 
	      resolutions SHALL also implicitly support a lower
resolution with 
	      the same MPI. 
	
	Roni Even 
	
	> >Some clarifications are as follows. 
	> > 
	> > 
	> > 
	> >>So far as I know (and I didn't recheck), H.263 only allows
for 
	> resolutions 
	> >>specified in the fmtp (though QCIF and CIF are I think
required). 
	If 
	> you 
	> >>want 344x288, you need to add a CUSTOM resolution to the
fmtp line. 
	> And 
	> >>the mpi must be an integer in the fmtp.  Now, if both sides
use 
	this, 
	> then 
	> >>those restrictions could be relaxed. 
	> > 
	> >In H.263, Custom Picture Format (CPFMT) is supported at level
50 or 
	> >higher. 
	> > 
	> >3GPP requires only level 45 of H.263 for its mobile video
telephony 
	> and 
	> >whether its level is increased further is doubtful. In other
words, 
	> MPEG-4 
	> >and H.264 are expected to utilize imageattr a lot. 
	>  
	fmtp 
	large 
	> >Sincerely yours, 
	> >Sender : Randell Jesup<[email-masked]> 
	> >Date : 2008-07-29 00:49 (GMT+09:00) 
	> >Title : Re: [MMUSIC] [AVT] spatial-resolution parameter for
RFC 
	> 3984bis 
	> > 
	> >"Ingemar Johansson S" <[email-masked]>
writes: 
	> >>To answer Roni's and Ye-Kui's comments I believe that the
image 
	> attribute 
	> >>can be used for 3. However, looking in perspective I am not
longer 
	> sure 
	> >>that the possibility to specify a framerate range makes
sense as it 
	> does 
	> >>not connect  too wellto reality when it comes to e.g H.264
which 
	> talks 
	> >>about macroblocks per second. The conclusion is then that
the 
	> framerate 
	> >>(or MBs) is determined from the fmtp line. 
	> >>This means that if the imageattr specifies a large offered
range the 
	> frame 
	> >>rate is limited by what is possible for the given preferred 
	> resolution. 
	> >> 
	> >>As an example, consider H.263 with the fmtp line 
	> >> CIF=4;QCIF=3;SQCIF=2; 
	> >>the imageattr may then be specified as 
	> >>a=imageattr:97 [x=[128:8:352],y=[96:8:288],par=[1.2,1.3]] 
	> >>If the receiver prefers the resolution 344x288 (slightly
less than 
	> >>If the prefered resolution is 160x128 (slightly less than
QCIF) then 
	> >So far as I know (and I didn't recheck), H.263 only allows
for 
	> resolutions 
	> >specified in the fmtp (though QCIF and CIF are I think
required).  If 
	> you 
	> >want 344x288, you need to add a CUSTOM resolution to the fmtp
line. 
	> And 
	> >the mpi must be an integer in the fmtp.  Now, if both sides
use this, 
	> then 
	> >those restrictions could be relaxed. 
	> > 
	> >>For the H.264 case the frame rate depending of the preferred
image 
	> size 
	> >>would be limited according to the maximum framerate
repending on the 
	> max 
	> >>MBs and the resolution given in table A-6 in the H.264 spec.
For 
	> instance 
	> >>for level 2 an image size of 344x280 which is less than CIF
but 
	> larger 
	> >>than QVGA would yield a max frame rate of 30Hz. 
	> > 
	> >My suggestion would be for this to express preferences within
the 
	> normal 
	> >negotiated resolution and framerate space.  i.e for 264 the
level and 
	> >max-fs and max-mbps would determine the *maximum* frame and
framerate 
	> >supported, but this would be used to decide what to actually
send. 
	> > 
	> >See my original email below -- look for: 
	> > "I would suggest instead (and did in the thread back in
March)" 
	> > 
	> >> 
	> >>Would this distincion between framerate (or MBs) and
preferred image 
	> >>/Ingemar 
	> >>> From: Even, Roni [mailto:[email-masked]] 
	> >>> Sent: den 28 juli 2008 11:32 
	> >>> To: [email-masked]; Ingemar Johansson S;
[email-masked] 
	> >>> Cc: [email-masked] 
	> >>> Subject: RE: [AVT] spatial-resolution parameter for RFC
3984bis 
	> >>> 
	> >>> Hi, 
	> >>> I think that we have to be clear about the usage of the
parameter 
	> >>> 
	> >>> 1. Announce sending of the specific resolution - just 
	> >>> resolution 2. Request a specific resolution - Receiver
wants 
	> >>> a specific resolution 3. Negotiate available resolutions -

	> >>> The offer says, I can do CIF, QCIF, 1080P,... here it may 
	> >>> dependent in H.263 and H.261 on the frame rate while in
H.264 
	> >>> the limit is in the macroblocks per second which will
specify 
	> >>> the maximum frame rate. The reason for this offer is
because 
	> >>> it offers the operation points that the offerer can have.
The 
	> >>> level parameter in H.263 and H.264 gives a maximum but
does 
	> >>> not imply support for any resolution that the answer would

	> >>> like to receive that falls within the range of the level. 
	> >>> 
	> >>> I think the img attribute address 1 and 2, I am not sure
about 3 
	> >>> > From: [email-masked]
[mailto:[email-masked]] 
	> >>> > Sent: Monday, July 28, 2008 1:11 PM 
	> >>> > To: [email-masked]; [email-masked] 
	> >>> > Cc: Even, Roni; [email-masked] 
	> >>> > Subject: RE: [AVT] spatial-resolution parameter for RFC
3984bis 
	> >>> > 
	> >>> > 
	> >>> > Interesting! I just wrote the following email, planned
to 
	> >>> send to the 
	> >>> > MMUSIC mailing list. Now this comes only to AVT. Feel
free 
	> >>> to forward 
	> >>> > it to MMUSIC (don't know whether it is allowed to cross
posting) 
	> or 
	> >>> > let me to send that, if needed. 
	> >>> > 
	> >>> > 
	> >>> > After a reading of this draft
(draft-johansson-mmusic-image- 
	> >>> > attributes), I think the sprop-spatial-resolution
parameter in 
	> the 
	> >>> > RFC3984bis can go. It is completely covered by
imageattr. 
	> >>> > A couple of comments to
draft-johansson-mmusic-image-attributes. 
	> >>> > 
	> >>> > - It would be helpful to introduce the methods in
payload 
	> >>> format specs 
	> >>> > for H.261, H.263 and H.264. In particular, for the one I
am 
	> >>> familiar 
	> >>> > with (i.e. H.264), I am not sure how exactly image size 
	> >>> together with 
	> >>> > frame rate is negotiated. 
	> >>> > 
	> >>> > - Re the following paragraph, I am not sure which
parameters are 
	> >>> > "similar features" and MUST NOT be specified in
imageattr. 
	> >>> Basically, 
	> >>> > H.264 sequence parameter set (SPS)may contain all 
	> >>> information relating 
	> >>> > to what may be included in imageattr according to the
curent 
	> draft. 
	> >>> > For example, the VUI part of SPS may even contain frame 
	> >>> rate related 
	> >>> > synatx elements. 
	> >>> >    o  Conflict with sprop-parameter-set: H.264 defines
the 
	sprop- 
	> >>> >       parameter-set which may define features that are 
	> >>> defined in this 
	> >>> >       draft.  To avoid conflicts, sessions that use the 
	imageattr 
	> >>> >       attribute MUST NOT specify similar features in 
	> >>> sprop-parameter- 
	> >>> >       set. 
	> >>> > 
	> >>> > BR, YK 
	> >>> > 
	> >>> > >From: ext Ingemar Johansson S 
	> >>> > >[mailto:[email-masked]] 
	> >>> > >Sent: Monday, July 28, 2008 1:04 PM 
	> >>> > >To: [email-masked] 
	> >>> > >Cc: Wang Ye-Kui (Nokia-NRC/Tampere); Even, Roni;
Randell Jesup 
	> >>> > >Subject: Re: [AVT] spatial-resolution parameter for RFC
3984bis 
	> >>> > > 
	> >>> > >Hi 
	> >>> > > 
	> >>> > > 
	> >>> > >Given the discussion below I really wonder if it is not
a 
	better 
	> >>> > >alternative to specifiy preferred image sizes using a
dedicated 
	> >>> > >(generic) SDP attribute. 
	> >>> > >Even though this is solved in some way in 3984bis we
still have 
	> to 
	> >>> > >provide with a solution for MPEG-4 or H.263 later on. 
	> >>> > >I drafted a proposed solution for an "a=imageattr"
attribute 
	> some 
	> >>> > >time ago that may be useful 
	> >>> > >http://tools.ietf.org/id/draft-johansson-mmusic-image- 
	> attributes- 
	> >>> > 01.txt 
	> >>> > >The draft provides with a solution that may work but
lacks a 
	> >>> > >requirement spec to outline exactly what the attribute 
	> >>> aims for. Also 
	> >>> > >extra work is needed to avoid conflicts with related 
	> >>> payload format 
	> >>> > >parameters. 
	> >>> > > 
	> >>> > >Regards 
	> >>> > >Ingemar 
	> >>> > > 
	> >>> > > 
	> >>> > >============================= 
	> >>> > >To: <Ye-Kui.Wang at nokia.com> 
	> >>> > >Subject: Re: [AVT] spatial-resolution parameter for RFC
3984bis 
	> >>> > >From: Randell Jesup <rjesup at wgate.com> 
	> >>> > >Date: Mon, 14 Jul 2008 09:35:18 -0400 
	> >>> > >Cc: roni.even at polycom.co.il, avt at ietf.org 
	> >>> > > 
	> >>> > ><Ye-Kui.Wang at nokia.com> writes: 
	> >>> > >>The resolution must be restricted by the level part of

	> >>> > >>profile-level-id and max-fs. Therefore, if a
configuration is 
	> agreed 
	> >>> > >>(or agreeable), then the answerer must request a
resolution 
	> under 
	> >>> > >>the constraint, and the offerer must accept the
request as 
	long 
	> as 
	> >>> > >>it meets the restriction.  This is similar as
sprop-parameter- 
	> sets, 
	> >>> > >>as long as it is complaint with the agreed (or
agreeable) 
	> >>> > >>configuration, the other side must accept it. 
	> >>> > > 
	> >>> > >I would suggest instead (and did in the thread back in
March): 
	> that 
	> >>> > >as a receiver property, it is the *preferred* maximum
frame 
	> size. 
	> >>> > >The receiver is required to decode and handle any size
up to 
	the 
	> size 
	> >>> > >implied by the level (and max-fs if given), but the
receiver 
	> prefers 
	> >>> > >this size (for example, it may support 1920x1080, but
only have 
	> an 
	> >>> > >LCD display that can display 1200x720, so there's no
*need* to 
	> send 
	> >>> > >it anything higher (and the encoder may prefer to avoid

	encoding 
	> more 
	> >>> > >macroblocks, especially in an MCU that shares
resources)).  In 
	> other 
	> >>> > >cases, the sender may have the stream already encoded
in 
	> different 
	> >>> > >resolutions, and this can help select the best one to
send to 
	> save 
	> >>> > >encoding (and bandwidth) resources. 
	> >>> > > 
	> >>> > >There is no requirement that the encoder scale, but if
it wants 
	> to it 
	> >>> > >can. 
	> >>> > > 
	> >>> > >It also (if I remember) provides aspect-ratio
information. 
	> >>> > >>Would it work this way? 
	> >>> > >>BR, YK 
	> >>> > >>>From: ext Even, Roni [mailto:roni.even at
polycom.co.il] 
	> >>> > >>>Sent: 14 July, 2008 10:38 
	> >>> > >>>To: Wang Ye-Kui (Nokia-NRC/Tampere); avt at ietf.org 
	> >>> > >>>Subject: RE: [AVT] spatial-resolution parameter for
RFC 
	> 3984bis 
	> >>> > >>> 
	> >>> > >>>YK, 
	> >>> > >>>Just to clarify, 
	> >>> > >>>The text on the receiver side says that this is the
requested 
	> >>> > >>>resolution, it does not address issue of negotiation,

	> >>> what will the 
	> >>> > >>>sender do if it cannot send this resolution, should
it 
	> >>> reject the 
	> >>> > >>>stream?. 
	> >>> > >>>If this is for negotiation do you except to do a new 
	> >>> offer answer 
	> >>> > >>>if the receiver wants a new resolution? 
	> >>> > >>> 
	> >>> > >>>Roni 
	> >>> > >>> 
	> >>> > >>>> From: avt-bounces at ietf.org [mailto:avt-bounces
at 
	> >>> ietf.org] On 
	> >>> > >>>> Behalf Of Ye-Kui.Wang at nokia.com 
	> >>> > >>>> Sent: Sunday, July 13, 2008 7:57 PM 
	> >>> > >>>> To: avt at ietf.org 
	> >>> > >>>> Subject: [AVT] spatial-resolution parameter for RFC
3984bis 
	> >>> > >>>> 
	> >>> > >>>> 
	> >>> > >>>> Concluded from the discussions in the email threads
listed 
	> >>> > >below, I 
	> >>> > >>>> added a new parameter, spatial-resolution, to the
RFC 
	> >>> > >3984bis draft 
	> >>> > >>>> (which will be submitted by tomorrow): 
	> >>> > >>>> 
	> >>> > >>>> http://www.ietf.org/mail- 
	> >>> > >>>> http://www.ietf.org/mail- 
	> >>> > >>>> The parameter is specified as follows (basically as
Randell 
	> >>> > >>>> suggested in 
http://www.ietf.org/mail-archive/web/avt/current/msg09429.html). 
	> >>> > >>>> 
	> >>> > >>>> spatial-resolution: 
	> >>> > >>>> This parameter MAY be used to indicate the maximum
spatial 
	> >>> > >>>> resolution of a NAL unit stream or the preferred
spatial 
	> >>> > >resolution 
	> >>> > >>>> of a receiver. 
	> >>> > >>>> The value is a base16 [6] (hexadecimal)
representation of 
	> the 
	> >>> > width 
	> >>> > >>>and 
	> >>> > >>>> height of the spatial resolution, in pixels,
separated 
	> >>> by a comma. 
	> >>> > >>>> 
	> >>> > >>>> Usage in SDP offer/answer: 
	> >>> > >>>> - When "a=sendrecv" or no direction attribute is
used, 
	> >>> > >>>> "spatial-resolution" indicates both the maximum
spatial 
	> >>> > >>>> resolution of the stream sent by the declaring
entity and 
	> the 
	> >>> > >preferred spatial 
	> >>> > >>>> resolution for receiving a stream. 
	> >>> > >>>> - When "a=sendonly", "spatial-resolution" indicates
the 
	> maximum 
	> >>> > >>>spatial 
	> >>> > >>>> resolution of the outgoing stream. 
	> >>> > >>>> - When "a=recvonly", "spatial-resolution" indicates

	> >>> the preferred 
	> >>> > >>>> spatial resolution for receiving a stream. 
	> >>> > >>>> 
	> >>> > >>>> Comments are welcome. 
	> >>> > >>>> 
	> >>> > >>>> BR, YK 
	> >>> > >>>> Audio/Video Transport Working Group avt at ietf.org

	> >>> > >>avt at ietf.org 
	> >>> > >OS team rjesup at wgate.com "The fetters imposed on 
	> >>> liberty at home 
	> >>> > >have ever been forged out of the weapons provided for 
	> >>> defence against 
	> >>> > >real, pretended, or imaginary dangers from abroad." 
	> >>> > >******************************************* 
	> >>> > >Ingemar Johansson 
	> >>> > >Senior Research Engineer, IETF "nethead" 
	> >>> > >EAB/TVP - Multimedia Technologies 
	> >>> > >Ericsson Research Ericsson AB 
	> >>> > >Box 920 S-971 28 Luleaa, Sweden 
	> >>> > >Tel: +46 (0)8 4043042 
	> >>> > >ECN: 850-43042 
	> >>> > >ECC: 850-43074 
	> >>> > >Mobile: +46 (0)730 783289 
	> >>> > >******************************************* 
	> >>> > > 
	> >>> 
	>  
	>  
	> -- 
	from

---

From: "Even, Roni" <[email-masked]>
Date: Wed, 30 Jul 2008 19:44:37 +0300

Ingemar,

ForH.264 the level parameter defines the maximum FS (frame size) in
macro blocks. The image can not be bigger than that since it specify the
maximum display buffer on the receiver. There is a parameter that
overrides it maxFS that allows the receiver to say that he can support
larger picture but then the level parameter maxmbps (max macro blocks
per second) will limit the frame rate. So For h.264 you cannot give
picture size bigger then level or maxFS

For H.263 you either specify maximum resolution using the parameters in
RFC 4629 or the level in table X.2 in H.263.

Text from H.263

Custom picture formats can have a custom pixel aspect ratio as described
in Table 3, if the custom pixel aspect ratio use is first negotiated by
external means. Custom picture formats can have any number of lines and
any number of pixels per line, provided that the number of lines is
divisible by four and is in the range [4, ... , 1152], and provided that
the number of pixels per line is also divisible by four and is in the
range [4, ... , 2048]. For picture formats having a width or height that
is not divisible by 16, the picture is decoded in the same manner as if
the width or height had the next larger size that would be divisible by
16 and then the picture is cropped at the right and the bottom to the
proper width and height for display purposes only.

Table 3/H.263 - Custom pixel aspect ratios

Pixel aspect ratio

Pixel width:Pixel height

Square

1:1

CIF

12:11

525-type for 4:3 picture

10:11

CIF for 16:9 picture

16:11

525-type for 16:9 picture

40:33

Extended PAR

m:n, m and n are relatively prime

All decoders and encoders shall be able to operate using the CIF picture
clock frequency. Some decoders and encoders may also support custom
picture clock frequencies. All decoders shall be able to operate using
the sub-QCIF picture format. All decoders shall also be able to operate
using the QCIF picture format. Some decoders may also operate with CIF,
4CIF or 16CIF, or custom picture formats. Encoders shall be able to
operate with one of the picture formats sub-QCIF and QCIF. The encoders
determine which of these two formats are used, and are not obliged to be
able to operate with both. Some encoders can also operate with CIF,
4CIF, 16CIF, or custom picture formats.

For H.261  there is no support for arbitrary picture size (CIF is the
largest) see RFC 4587

So in the general case there should be a way to reject the request and
maybe give the supported resolution, still it will be wrong to use this
attribute for H.261

Roni

From: Ingemar Johansson S [mailto:[email-masked]] 
Sent: Wednesday, July 30, 2008 12:23 PM
To: [email-masked]; Even, Roni
Cc: Randell Jesup; [email-masked]; [email-masked]; [email-masked]
Subject: RE: RE: [MMUSIC] [AVT] spatial-resolution parameter for RFC
3984bis

Hi

/Ingemar

---

From: Randell Jesup <[email-masked]>
Date: Wed, 30 Jul 2008 14:46:16 -0400

"Ingemar Johansson S" <[email-masked]> writes:
>So if I understand the H.264 part correctly the H.264 level-id parameter
>will provide with a upper limit to the maximum image size regardless of
>what is specified by imageattr, maxFS does infact specify something that
>relates to imageattr so here there may be a conflict.
>
>I believe that it makes sense to let "level-id in H.264" and "H.263 max
>resolution" be the limiting factor for imageattr as it will otherwise be
>too complex to incorporate hardware awareness into the attribute. 
>Under normal circumstances the offer will not specify an imageattr that
>goes beyond e.g the level-id but such cases may occur if one for
>instance specify the attribute in the SDPMedCapNeg framework.

My suggestion (again) is that codec-specific parameters limiting frame size
and frame rate are never overridden by imageattr.  So H.264 level (and
max-fs/max-mbps) specify the maximums allowed, and H.263 fmtp lines with
resolutions and MPIs specify which resolutions are allowed (perhaps
including CUSTOM).  The imageattr attribute is used to decide, within those
limits, what to *actually* send by exposing details of the receiver's
capabilities and preferences.  It extends and/or replaces the rough
preferences provided by the H.263 RFCs where order of declaration gave a
preference, for example.

Other issues:
>From the spec:
       a=imageattr:<PT> 1*WSP <sar=range 1*WSP> <list>
       list = set *(1*WSP set) ; defined in RFC 4566

       <PT> is the payload type number
       sar is the a set of supported sample aspect ratios (optional)

Can we allow "*" for payload type, like a=rtcp-fb does?  This could
be very handy to avoid an explosion of lines.

   A new "imageattr" attribute is defined.  The imageattr attribute
   contains a set of different image-resolution options that the offerer
   can provide.  The receiver can then select the desired image
   attribute (e.g image size) and has then the ability to avoid costly
   transformations (e.g rescaling) of the images.  In this approach only
   the image resolution and optionally the framerate, sample aspect
   ratio and preference is covered but the framework makes it possible
   to extend with other image related attributes that makes sense.

Um, a significant problem here.  There seems to be a requirement
that the negotiation is one-directional.  In this case, the offerer
provides NO indication about what he prefers to receive, just what he *can*
send.  When the answerer responds "picking" a resolution, what does the
answerer send?  The implication is that it's the same as what they picked
to receive, which may be seriously sub-optimal.

This appears to be handled by giving the "incomplete" offer of
a=imageattr:97 *
and having the answerer fill in the imageattr in the response.  This still
requires a symmetric session (identical resolutions, frame rates, etc).

The imageattr spec "solves" this by requiring use of two RTP sessions; one
in each direction.  In theory this works, but it's a painful way to solve
the problem, and many devices will not handle this formulation.  It's also
odd to force this, since you're not even required to use the same codecs in
the two directions, but here we're forcing two sessions just to support
different resolutions or framerates.

Even worse, the offerer has to "guess" if the negotiation will end up
asymmetric, and offer two sessions from the start.  The answerer can't
add sessions, let alone split them.  Plus this is relying on implicit
grouping; there's nothing specifying that both sessions are expected to
be active together - many will accept one and reject the other.

If you need separate "send" and "receive" formulations, then provide
them in one session.  Either:

a=imageattr-send:97 * (or list)
a=imageattr-receive:97 <list>

Or work send and receive into the syntax of imageattr directly (probably 
preferable).

a=imageattr:97 send * receive <list>
or
a=imageattr:97 send <list> receive <list>

-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS team
[email-masked]
"The fetters imposed on liberty at home have ever been forged out of the weapons
provided for defence against real, pretended, or imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)

---

From: "Ingemar Johansson S" <[email-masked]>
Date: Thu, 31 Jul 2008 10:50:16 +0200

Hi

Thanks for the comments, my cmments inline below.

/Ingemar

> Sent: den 30 juli 2008 19:46
> To: Ingemar Johansson S
> Cc: Even, Roni; [email-masked]; [email-masked]; 
> Subject: Re: [MMUSIC] [AVT] spatial-resolution parameter for 
> RFC 3984bis
> 
> >So if I understand the H.264 part correctly the H.264 level-id 
> >parameter will provide with a upper limit to the maximum image size 
> >regardless of what is specified by imageattr, maxFS does 
> infact specify 
> >something that relates to imageattr so here there may be a conflict.
> >
> >resolution" be the limiting factor for imageattr as it will 
> otherwise 
> >be too complex to incorporate hardware awareness into the attribute.
> My suggestion (again) is that codec-specific parameters 
> limiting frame size and frame rate are never overridden by 
> imageattr.  So H.264 level (and
> max-fs/max-mbps) specify the maximums allowed, and H.263 fmtp 
> lines with resolutions and MPIs specify which resolutions are 
> allowed (perhaps including CUSTOM).  The imageattr attribute 
> is used to decide, within those limits, what to *actually* 
> send by exposing details of the receiver's capabilities and 
> preferences.  It extends and/or replaces the rough 
> preferences provided by the H.263 RFCs where order of 
> declaration gave a preference, for example.

I believe I agree (even though the formulation in the draft may say
different). A reasonable assumption is that the codec specific
parameters are the limiting factor.

> Can we allow "*" for payload type, like a=rtcp-fb does?  This 
> could be very handy to avoid an explosion of lines.

Didn't think of it but I see no problem in that esp. with the assumtion
above that the codec specific parameters are the limiting factor.

> Um, a significant problem here.  There seems to be a 
> requirement that the negotiation is one-directional.  In this 
> case, the offerer provides NO indication about what he 
> prefers to receive, just what he *can* send.  When the 
> answerer responds "picking" a resolution, what does the 
> answerer send?  The implication is that it's the same as what 
> they picked to receive, which may be seriously sub-optimal.
> 
> and having the answerer fill in the imageattr in the 
> response.  This still requires a symmetric session (identical 
> resolutions, frame rates, etc).
> 
> The imageattr spec "solves" this by requiring use of two RTP 
> sessions; one in each direction.  In theory this works, but 
> it's a painful way to solve the problem, and many devices 
> will not handle this formulation.  It's also odd to force 
> this, since you're not even required to use the same codecs 
> in the two directions, but here we're forcing two sessions 
> just to support different resolutions or framerates.
> 
> Even worse, the offerer has to "guess" if the negotiation 
> will end up asymmetric, and offer two sessions from the 
> start.  The answerer can't add sessions, let alone split 
> them.  Plus this is relying on implicit grouping; there's 
> nothing specifying that both sessions are expected to be 
> active together - many will accept one and reject the other.
> 
> If you need separate "send" and "receive" formulations, then 
> provide them in one session.  Either:
> 
> Or work send and receive into the syntax of imageattr 
> directly (probably preferable).
> 

Actually this is one of the parts in the draft that I don't like and I
planned to ask the group for suggestions how to improve the assymetric
setup. Your alternative suggestions look attractive.
One reason to why I initially picked the two RTP session alternative as
one may anyway want to run different codecs in the two directions but as
you said it may cause a lot of trouble.

> 
> --
> Randell Jesup, Worldgate (developers of the Ojo videophone), 
> ex-Amiga OS team [email-masked] "The fetters imposed on 
> liberty at home have ever been forged out of the weapons

---

From: Randell Jesup <[email-masked]>
Date: Thu, 31 Jul 2008 09:57:18 -0400

"Ingemar Johansson S" <[email-masked]> writes:
>> My suggestion (again) is that codec-specific parameters limiting frame
>> size and frame rate are never overridden by imageattr.  So H.264 level
>> (and max-fs/max-mbps) specify the maximums allowed, and H.263 fmtp lines
>> with resolutions and MPIs specify which resolutions are allowed (perhaps
>> including CUSTOM).  The imageattr attribute is used to decide, within
>> those limits, what to *actually* send by exposing details of the
>> receiver's capabilities and preferences.  It extends and/or replaces the
>> rough preferences provided by the H.263 RFCs where order of declaration
>> gave a preference, for example.
>

Good.

>above that the codec specific parameters are the limiting factor.

I think that works well, since often the imageattr will specify the
physical resolution or optimum display resolution, plus some preferred
scaled (or unscaled) sizes, and these often will be codec-invariant.

>setup. Your alternative suggestions look attractive.

Thanks.

>you said it may cause a lot of trouble.

You certainly can run different codecs in each direction, or the same codec
initially but switch codecs (or switch resolutions) on the fly with no
renegotiation, assuming the offer and answer allow for it.  imageattr
shouldn't restrict these possibilities.