Subject: Re: [AVT] spatial-resolution parameter for RFC 3984bis

From: "Ingemar Johansson S" <[email-masked]>
Date: Wed, 16 Jul 2008 14:25:41 +0200

Hi

FYI
Related to this discussion is also the discussion around the imageattr
attribute in the MMUSIC WG. 
See 
http://www.ietf.org/mail-archive/web/mmusic/current/msg06980.html
for a discussion.

Regards
Ingemar

> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Sun, 13 Jul 2008 19:56:59 +0300
> From: <[email-masked]>
> Subject: [AVT] spatial-resolution parameter for RFC 3984bis
> To: <[email-masked]>
> Message-ID:
> 	<[email-masked]>
> Content-Type: text/plain;	charset="us-ascii"
> 
> 
> Concluded from the discussions in the email threads listed 
> below, I added a new parameter, spatial-resolution, to the 
> RFC 3984bis draft (which will be submitted by tomorrow): 
>  
> http://www.ietf.org/mail-archive/web/avt/current/msg09425.html
> http://www.ietf.org/mail-archive/web/avt/current/msg09650.html
> 
> The parameter is specified as follows (basically as Randell 
> suggested in 
> http://www.ietf.org/mail-archive/web/avt/current/msg09429.html). 
> 
> spatial-resolution:
> This parameter MAY be used to indicate the maximum spatial 
> resolution of a NAL unit stream or the preferred spatial 
> resolution of a receiver.
> The value is a base16 [6] (hexadecimal) representation of the 
> width and height of the spatial resolution, in pixels, 
> separated by a comma. 
> 
> Usage in SDP offer/answer: 
> - When "a=sendrecv" or no direction attribute is used, 
> "spatial-resolution" indicates both the maximum spatial 
> resolution of the stream sent by the declaring entity and the 
> preferred spatial resolution for receiving a stream. 
> - When "a=sendonly", "spatial-resolution" indicates the 
> maximum spatial resolution of the outgoing stream.
> - When "a=recvonly", "spatial-resolution" indicates the 
> preferred spatial resolution for receiving a stream.
> 
> Comments are welcome.
> 
> BR, YK
> 
> 
> ------------------------------
> 
> _______________________________________________
> Audio/Video Transport Working Group
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt
> 
> 
> End of avt Digest, Vol 51, Issue 19
> ***********************************
> 
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt

---

From: Kyunghun Jung <[email-masked]>
Date: Tue, 29 Jul 2008 06:27:09 +0000 (GMT)

Dear experts of IETF MMUSIC and AVT:

Let me add a few comments from 3GPP's point of view.

In fact, we asked MMUSIC to build "Imageattr" for 3GPP's IMS-based
video telephony.

(1) Codec Dependency

We need a generic, codec-independend mechanism  to negotiate image
size between mobile phones.

Parameter sets of recent codecs include the information
on image size but they are not considered during session

setup. How to couple imageattr and config (for MPEG-4)
or sprop-parameter-set (H.264) will be the key issue

in the session setup. To reduce session setup delay, the
offerer might include multiple lines of parameter sets

in SDP offer (one set for for each image size). It might
look quite messy.

(2) Frame Rate

For mobile video telephony like 3GPP's, frame rate is not negotiated but
decided by the sender after the bit-rate

and image size are fixed. Note also that even in wired video telephony, H.245
videoTemporalSpatialTradeoff or "a=quality"

works properly, only between proprietary terminals that know the far-end's
capability and codec configuration completely.

However, for us, frame rate is also a parameter to be controlled during communication.
When there is congestion in downlink

(base station to mobile) or somewhere else, receiving mobile asks the sending
mobile to reduce the bit-rate, which can be

executed by reducing the frame rate.

(3) For Mr. Even Roni

1. Announce sending of the specific resolution -
just resolution 2. Request a specific resolution - Receiver wants

a specific
resolution 3. Negotiate available resolutions - I think the img attribute address 1 and
2, I am not sure about 3

Thank you for the comment. For 3GPP, 3 is the key reason why we need imageattr.

If 3 is still unclear to any MMUSIC/AVT experts, immediate action will be
necessary to fix this issue during IETF '72.

Based on the feedback from IETF, 3GPP SA4 will cover imageattr more in depth
from August 18.

We will appreciate if you point out the area you are still not sure
yet.

Other comments and questions on imageattr will be greatly welcomed and answered.

We would like to ask experts of MMUSIC and AVT to discuss "a=imageattr"
with special interest, during IETF '72.

Sincerely yours,

Kyunghun Jung

Samsung Electronics Co., Ltd.

------- Original Message -------
Sender : Ingemar Johansson S<[email-masked]>
Date : 2008-07-28 22:01 (GMT+09:00)
Title : Re: [MMUSIC] [AVT] spatial-resolution parameter for RFC 3984bis

Hi

My response below, also a crosspost to mmusic, hope that this not too upsetting.

To answer Roni's and Ye-Kui's comments I believe that the image attribute can be used for 3. However, looking in perspective I am not longer sure that the possibility to specify a framerate range makes sense as it does not connect  too wellto reality when it comes to e.g H.264 which talks about macroblocks per second. The conclusion is then that the framerate (or MBs) is determined from the fmtp line.

This means that if the imageattr specifies a large offered range the frame rate is limited by what is possible for the given preferred resolution.

As an example, consider H.263 with the fmtp line

CIF=4;QCIF=3;SQCIF=2;

the imageattr may then be specified as

a=imageattr:97 [x=[128:8:352],y=[96:8:288],par=[1.2,1.3]]

If the receiver prefers the resolution 344x288 (slightly less than CIF) the maximum frame rate would be 30/(1.0001*4) ~ 7.5Hz.

If the prefered resolution is 160x128 (slightly less than QCIF) then the max frame rate is 30/(1.0001*3) Hz and so on.

For the H.264 case the frame rate depending of the preferred image size would be limited according to the maximum framerate repending on the max MBs and the resolution given in table A-6 in the H.264 spec. For instance for level 2 an image size of 344x280 which is less than CIF but larger than QVGA would yield a max frame rate of 30Hz.

Would this distincion between framerate (or MBs) and preferred image size work?

/Ingemar

> -----Original Message-----

> From: Even, Roni [mailto:[email-masked]]

> Sent: den 28 juli 2008 11:32

> To: [email-masked]; Ingemar Johansson S; [email-masked]

> Cc: [email-masked]

> Subject: RE: [AVT] spatial-resolution parameter for RFC 3984bis

> Hi,

> I think that we have to be clear about the usage of the parameter

> 1. Announce sending of the specific resolution - just

> resolution 2. Request a specific resolution - Receiver wants

> a specific resolution 3. Negotiate available resolutions -

> The offer says, I can do CIF, QCIF, 1080P,... here it may

> dependent in H.263 and H.261 on the frame rate while in H.264

> the limit is in the macroblocks per second which will specify

> the maximum frame rate. The reason for this offer is because

> it offers the operation points that the offerer can have. The

> level parameter in H.263 and H.264 gives a maximum but does

> not imply support for any resolution that the answer would

> like to receive that falls within the range of the level.

> I think the img attribute address 1 and 2, I am not sure about 3

> Roni Even

> > -----Original Message-----

> > From: [email-masked] [mailto:[email-masked]]

> > Sent: Monday, July 28, 2008 1:11 PM

> > To: [email-masked]; [email-masked]

> > Cc: Even, Roni; [email-masked]

> > Subject: RE: [AVT] spatial-resolution parameter for RFC 3984bis

> >

> >

> > Interesting! I just wrote the following email, planned to

> send to the

> > MMUSIC mailing list. Now this comes only to AVT. Feel free

> to forward

> > it to MMUSIC (don't know whether it is allowed to cross posting) or

> > let me to send that, if needed.

> >

> >

> > After a reading of this draft (draft-johansson-mmusic-image-

> > attributes), I think the sprop-spatial-resolution parameter in the

> > RFC3984bis can go. It is completely covered by imageattr.

> >

> > A couple of comments to draft-johansson-mmusic-image-attributes.

> >

> > - It would be helpful to introduce the methods in payload

> format specs

> > for H.261, H.263 and H.264. In particular, for the one I am

> familiar

> > with (i.e. H.264), I am not sure how exactly image size

> together with

> > frame rate is negotiated.

> >

> > - Re the following paragraph, I am not sure which parameters are

> > "similar features" and MUST NOT be specified in imageattr.

> Basically,

> > H.264 sequence parameter set (SPS)may contain all

> information relating

> > to what may be included in imageattr according to the curent draft.

> > For example, the VUI part of SPS may even contain frame

> rate related

> > synatx elements.

> >    o  Conflict with sprop-parameter-set: H.264 defines the sprop-

> >       parameter-set which may define features that are

> defined in this

> >       draft.  To avoid conflicts, sessions that use the imageattr

> >       attribute MUST NOT specify similar features in

> sprop-parameter-

> >       set.

> >

> > BR, YK

> >

> > >-----Original Message-----

> > >From: ext Ingemar Johansson S

> > >[mailto:[email-masked]]

> > >Sent: Monday, July 28, 2008 1:04 PM

> > >To: [email-masked]

> > >Cc: Wang Ye-Kui (Nokia-NRC/Tampere); Even, Roni; Randell Jesup

> > >Subject: Re: [AVT] spatial-resolution parameter for RFC 3984bis

> > >

> > >Hi

> > >

> > >

> > >Given the discussion below I really wonder if it is not a better

> > >alternative to specifiy preferred image sizes using a dedicated

> > >(generic) SDP attribute.

> > >Even though this is solved in some way in 3984bis we still have to

> > >provide with a solution for MPEG-4 or H.263 later on.

> > >I drafted a proposed solution for an "a=imageattr" attribute some

> > >time ago that may be useful

> > >http://tools.ietf.org/id/draft-johansson-mmusic-image-attributes-

> > 01.txt

> > >The draft provides with a solution that may work but lacks a

> > >requirement spec to outline exactly what the attribute

> aims for. Also

> > >extra work is needed to avoid conflicts with related

> payload format

> > >parameters.

> > >

> > >Regards

> > >Ingemar

> > >

> > >

> > >=============================

> > >To: <Ye-Kui.Wang at nokia.com>

> > >Subject: Re: [AVT] spatial-resolution parameter for RFC 3984bis

> > >From: Randell Jesup <rjesup at wgate.com>

> > >Date: Mon, 14 Jul 2008 09:35:18 -0400

> > >Cc: roni.even at polycom.co.il, avt at ietf.org

> > >

> > ><Ye-Kui.Wang at nokia.com> writes:

> > >>The resolution must be restricted by the level part of

> > >profile-level-id

> > >>and max-fs. Therefore, if a configuration is agreed (or

> agreeable),

> > >>then the answerer must request a resolution under the constraint,

> > >>and the offerer must accept the request as long as it meets the

> > >restriction.

> > >>This is similar as sprop-parameter-sets, as long as it is

> complaint

> > >>with the agreed (or agreeable) configuration, the other side

> > >must accept it.

> > >

> > >I would suggest instead (and did in the thread back in March):

> > >that as a receiver property, it is the *preferred* maximum frame

> > >size.  The receiver is required to decode and handle any

> size up to

> > >the size implied by the level (and max-fs if given), but

> the receiver

> > >prefers this size (for example, it may support 1920x1080, but only

> > >have an LCD display that can display 1200x720, so there's

> no *need*

> > >to send it anything higher (and the encoder may prefer to avoid

> > >encoding more macroblocks, especially in an MCU that shares

> > >resources)).  In other cases, the sender may have the

> stream already

> > >encoded in different resolutions, and this can help select

> the best

> > >one to send to save encoding (and bandwidth) resources.

> > >

> > >There is no requirement that the encoder scale, but if it

> wants to it

> > >can.

> > >

> > >It also (if I remember) provides aspect-ratio information.

> > >

> > >>Would it work this way?

> > >>

> > >>BR, YK

> > >>

> > >>>-----Original Message-----

> > >>>From: ext Even, Roni [mailto:roni.even at polycom.co.il]

> > >>>Sent: 14 July, 2008 10:38

> > >>>To: Wang Ye-Kui (Nokia-NRC/Tampere); avt at ietf.org

> > >>>Subject: RE: [AVT] spatial-resolution parameter for RFC 3984bis

> > >>>

> > >>>YK,

> > >>>Just to clarify,

> > >>>The text on the receiver side says that this is the requested

> > >>>resolution, it does not address issue of negotiation,

> what will the

> > >>>sender do if it cannot send this resolution, should it

> reject the

> > >>>stream?.

> > >>>If this is for negotiation do you except to do a new

> offer answer

> > >>>if the receiver wants a new resolution?

> > >>>

> > >>>Roni

> > >>>

> > >>>> -----Original Message-----

> > >>>> From: avt-bounces at ietf.org [mailto:avt-bounces at

> ietf.org] On

> > >>>> Behalf Of Ye-Kui.Wang at nokia.com

> > >>>> Sent: Sunday, July 13, 2008 7:57 PM

> > >>>> To: avt at ietf.org

> > >>>> Subject: [AVT] spatial-resolution parameter for RFC 3984bis

> > >>>>

> > >>>>

> > >>>> Concluded from the discussions in the email threads listed

> > >below, I

> > >>>> added a new parameter, spatial-resolution, to the RFC

> > >3984bis draft

> > >>>> (which will be submitted by tomorrow):

> > >>>>

> > >>>> http://www.ietf.org/mail-archive/web/avt/current/msg09425.html

> > >>>> http://www.ietf.org/mail-archive/web/avt/current/msg09650.html

> > >>>>

> > >>>> The parameter is specified as follows (basically as Randell

> > >>>> suggested in

> > >>>>

> > >>>>

> > >>>> spatial-resolution:

> > >>>> This parameter MAY be used to indicate the maximum spatial

> > >>>> resolution of a NAL unit stream or the preferred spatial

> > >resolution

> > >>>> of a receiver.

> > >>>> The value is a base16 [6] (hexadecimal) representation of the

> > width

> > >>>and

> > >>>> height of the spatial resolution, in pixels, separated

> by a comma.

> > >>>>

> > >>>> Usage in SDP offer/answer:

> > >>>> - When "a=sendrecv" or no direction attribute is used,

> > >>>> "spatial-resolution" indicates both the maximum spatial

> > >>>> resolution of the stream sent by the declaring entity and the

> > >preferred spatial

> > >>>> resolution for receiving a stream.

> > >>>> - When "a=sendonly", "spatial-resolution" indicates the maximum

> > >>>spatial

> > >>>> resolution of the outgoing stream.

> > >>>> - When "a=recvonly", "spatial-resolution" indicates

> the preferred

> > >>>> spatial resolution for receiving a stream.

> > >>>>

> > >>>> Comments are welcome.

> > >>>>

> > >>>> BR, YK

> > >>>> _______________________________________________

> > >>>> Audio/Video Transport Working Group avt at ietf.org

> > >>>> https://www.ietf.org/mailman/listinfo/avt

> > >>>

> > >>_______________________________________________

> > >>Audio/Video Transport Working Group

> > >>avt at ietf.org

> > >>https://www.ietf.org/mailman/listinfo/avt

> > >

> > >

> > >--

> > >Randell Jesup, Worldgate (developers of the Ojo

> videophone), ex-Amiga

> > >OS team rjesup at wgate.com "The fetters imposed on

> liberty at home

> > >have ever been forged out of the weapons provided for

> defence against

> > >real, pretended, or imaginary dangers from abroad."

> > >        - James Madison, 4th US president (1751-1836)

> > >

> > >*******************************************

> > >Ingemar Johansson

> > >Senior Research Engineer, IETF "nethead"

> > >EAB/TVP - Multimedia Technologies

> > >Ericsson Research Ericsson AB

> > >Box 920 S-971 28 Luleå, Sweden

> > >Tel: +46 (0)8 4043042

> > >ECN: 850-43042

> > >ECC: 850-43074

> > >Mobile: +46 (0)730 783289

> > >*******************************************

> > >

mmusic mailing list

[email-masked]

https://www.ietf.org/mailman/listinfo/mmusic

---

From: Kyunghun Jung <[email-masked]>
Date: Tue, 29 Jul 2008 06:16:51 +0000 (GMT)

(1) Codec Dependency

(2) Frame Rate

(3) For Mr. Even Roni

a specific

Sincerely yours,

Kyunghun Jung

------- Original Message -------
Sender : Ingemar Johansson S<[email-masked]>
Date : 2008-07-28 22:01 (GMT+09:00)
Title : Re: [MMUSIC] [AVT] spatial-resolution parameter for RFC 3984bis

Hi

CIF=4;QCIF=3;SQCIF=2;

/Ingemar

mmusic mailing list

[email-masked]

---

From: Kyunghun Jung <[email-masked]>
Date: Wed, 30 Jul 2008 00:01:12 +0000 (GMT)

Thank you Mr. Even Roni for the information.

We would like to ask experts in MMUSIC and AVT to pay a special interest
to the relationship

between imageattr and config/sprop-parameter-set. How to stack the set of
information in the SDP offer

will influence the session setup.

For example, suppose imageattr specifies image sizes over multiple levels.
The following form might be necessary.

m=video 49154 RTP/AVPF 99

b=AS:48

b=RS:0

b=RR:2500

a=rtpmap:99 MP4V-ES/90000

a=imageattr:99 [x=x1,y=y1] [x=x2,y=y2] [x=x3,y=y3]

a=fmtp:99 profile-level-id=a; \
config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1

a=fmtp:99 profile-level-id=b; \

config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_2

a=fmtp:99 profile-level-id=c; \

config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_3

In case imageattr specifies image sizes within a single level.

a=imageattr:99 [x=x1,y=y1] [x=x2,y=y2] [x=x3,y=y3]

a=fmtp:99 profile-level-id=8

a=fmtp:99 config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1

a=fmtp:99 config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_2

a=fmtp:99 config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_3

The above forms are only basic possibilities and more systematic design and
suggestion are

requested from MMUSIC and AVT experts.

Have a nice meeting at Dublin!

Kyunghun Jung

------- Original Message -------
Sender : Even, Roni<[email-masked]>
Date : 2008-07-30 00:13 (GMT+09:00)
Title : RE: [MMUSIC] [AVT] spatial-resolution parameter for RFC 3984bis

Hi,

First of all, ITU-T documents in pdf format are free at

http://www.itu.int/rec/T-REC/e

As for custom format inH.263. RFC 4629 section 8.1.1 had the custom

resolution, no relation to specific level.

CUSTOM: Specifies the MPI (Minimum Picture Interval) for a

custom-defined resolution.  The custom parameter receives three

comma-separated values, Xmax, Ymax, and MPI.  The Xmax and Ymax

parameters describe the number of pixels in the X and Y axis and

must be evenly divisible by 4.  The permissible values for MPI are

integer values from 1 to 32, which correspond to a maximum frame

rate of 30/(1.001 *the specified value).

Also note the next sentence

A system that declares support of a specific MPI for one of the

resolutions SHALL also implicitly support a lower resolution with

the same MPI.

Roni Even

> From: Randell Jesup [mailto:[email-masked]]

> Sent: Tuesday, July 29, 2008 5:08 PM

> To: [email-masked]

> Cc: Ingemar Johansson S; [email-masked]; Even, Roni; Ye-

> [email-masked]; [email-masked]

> Subject: Re: [MMUSIC] [AVT] spatial-resolution parameter for RFC

> 3984bis

> Kyunghun Jung <[email-masked]> writes:

> >Some clarifications are as follows.

> >>So far as I know (and I didn't recheck), H.263 only allows for

> resolutions

> >>specified in the fmtp (though QCIF and CIF are I think required).

If

> you

> >>want 344x288, you need to add a CUSTOM resolution to the fmtp line.

> And

> >>the mpi must be an integer in the fmtp.  Now, if both sides use

this,

> then

> >>those restrictions could be relaxed.

> >In H.263, Custom Picture Format (CPFMT) is supported at level 50 or

> >higher.

> >3GPP requires only level 45 of H.263 for its mobile video telephony

> and

> >whether its level is increased further is doubtful. In other words,

> MPEG-4

> >and H.264 are expected to utilize imageattr a lot.

> I think you misunderstand my comment.  I was referring the the SDP

fmtp

> line for H.263, where custom resolutions (other then CIF, etc) can be

> specified.

> Since the ITU specs are a pain to come by (I think I have an old H.263

> and

> Annexes stashed somewhere, but it isn't handy at the moment, from

> home), I

> can't easily look up the Annex X profile/level stuff; I just know how

> the

> SDP works from the IETF spec.  (I tried to look it up online - Ugh,

> requires TIES account.  The ITU appears to not want anyone except

large

> companies to implement their specs.)  :-(

> It sounds like "CUSTOM" isn't of any use for many scenarios.

> In any case, I was responding to Ingemar's example that used custom

> resolutions in H.263.

> >Sincerely yours,

> >Kyunghun Jung