Subject: Re: [MMUSIC] imageattr and codec parameter sets (with embedded image size info)

From: Kyunghun Jung <[email-masked]>
Date: Thu, 31 Jul 2008 10:37:27 +0000 (GMT)

Dear Mr.
Johansson:

Thank you for the clarifications.

Then we might also need to comment on MPEG-4's config, which does similar
roles as sprop-parameter-sets of H.264.

In addition, config includes image size information too. P. 38 of 3GPP TS
26.234 says:

The frame size field in SDP is needed by the client in order to properly
allocate frame buffer memory.  For MPEG-4
Visual streams, the frame size shall be extracted from the "config"
information in the SDP. For H.264
(AVC) streams, the frame size
shall be extracted from the sprop-parameters-sets information in the SDP. For H.263 streams, a PSS server shall include the
"a=framesize" field at the media level for each stream in SDP, and a PSS
client should interpret this field, if present. Clients should be ready to
receive SDP descriptions without
this attribute.

We might need to check whether other bitstream of config or sprop-parameter-sets
does not change after the size information is removed,

i.e., we need to check whether one line of config or sprop-parameter-sets
can cover multiple image sizes. If so, then the format of imageattr

will look like

m=video 49154 RTP/AVPF 99

b=AS:48

b=RS:0

b=RR:2500

a=rtpmap:99 MP4V-ES/90000

a=fmtp:99 profile-level-id=a; \
config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1

a=imageattr:99 [x=x1,y=y1] [x=x2,y=y2]
[x=x3,y=y3]

...

I will confirm these with other experts.

By the way, specifying the largest profile and level will do the job but
maintaining a minimum one might will also save resources

in the decoder. The following is just an implementation issue and do not
consider seriously: if level 1.2 is negotiated but

the imageattr negotiation results in QCIF. Which video
decoder should be setup inside a mobile? QCIF or CIF? This is also

an interesting question and I think many MPEG colleagues will follow profile
and level.  Of course, we mobile phone guys will follow

imageattr. (^^)

Sincerely yours,

Kyunghun Jung

------- Original Message -------
Sender : Ingemar Johansson S<[email-masked]>
Date : 2008-07-31 18:56 (GMT+09:00)
Title : RE: imageattr and codec parameter sets (with embedded image size info)

Hi

My
assumption here is that:

while the
profile-level-id is the limiting factor for imageattr

the
pic_width_in_mbs_minus_1 and pic_height_in_map_units_minus_1 in
sprop-parameter-sets should not be used if imageattr is
used.

This is
expressed in the first bullet in the section 5 in the draft.

That
said, I am not 100% sure that I adressed your issue
below.

As regards
to the "q value" coment below, unfortunately it does not address Randell's issue
below.

Regards

/Ingemar

From: Kyunghun Jung
[mailto:[email-masked]]
Sent: den 31 juli 2008
06:38
To: Ingemar Johansson S
Cc: Even, Roni; Randell
Jesup; [email-masked]; [email-masked]
Subject: imageattr and codec
parameter sets (with embedded image size info)

Dear Mr. Johannson, Mr. Even Roni, Mr. Randell Jesup and other
experts of MMUSIC/AVT:

The draft of imageattr currently does not introduce the codec parameter set
issues, which also have image size information.

For example, sprop-parameter-sets contains
pic_width_in_mbs_minus1 and pic_height_in_map_units_minus_1.

Let me summarize.

- profile and level specify the maximum image
size

- imageattr specify the image sizes available from
sender

- config/sprop-parameters-sets specify the image
size to be encoded (but not considered in session setup)

Therefore I asked your opinions on the forming of SDP offer and suggested
three (crudely drafted) examples of

imageattr coupled with config/sprop-parameter-sets. In fact, one form
showed a possibility that covers multiple levels.

If imageattr is limited to a level, then the following form might be
necessary to match the sizes in imageattr with three

sets of config with different image size. My question was
whether this form could be acceptable?

a=imageattr:99 [x=x1,y=y1] [x=x2,y=y2]
[x=x3,y=y3]

a=fmtp:99 profile-level-id=8

a=fmtp:99 config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1

a=fmtp:99 config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_2

a=fmtp:99 config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_3

Or do we need to send config after the image size is
decided? This will increase the session setup delay.

We are fine with either one imageattr for one profile/level or one
imageattr for multiple profile/levels.

But each possibility of handling parameter sets will influence the format
of imageattr. Therefore we asked

experts in IETF to clarify this issue.

Let me answer a question from Mr. Randell Jesup.

Um, a significant
problem here.
There seems to be a requirement that the
negotiation is one-directional.  In this case, the offerer
provides NO indication about what he prefers to receive, just what he *can*
send.
When the answerer responds "picking" a
resolution, what does the answerer send?  The implication is that it's the same as what they picked to
receive, which may be seriously sub-optimal.

-> q value, as in Example 3 of the draft, will offer the priority
among offered sizes.

Sincerely yours,

Kyunghun Jung

------- Original Message -------
Sender : Ingemar
Johansson S<[email-masked]>
Date :
2008-07-30 18:23 (GMT+09:00)
Title : RE: RE: [MMUSIC] [AVT]
spatial-resolution parameter for RFC 3984bis

Hi

If I understand your email
correctly we will, with the constructed SDP's below, have the potential issue
that imageattr may offer combinations (x and y resoluions) that are not
allowed for a given profile level ID. The solution I see is that the profile
levels provides with an upper (maybe also a lower?) limit.

For instance if we have some
(really wierd) imageattr like

a=imageattr:97
[x=[128:16:1920], y=[96:16:1280],par=[1.1 1.3]]

and the profile levels only
supports a resolution up to 352x288 , the latter limitation will apply which
means that the imageattr should infact be interpreted as

a=imageattr:97 [x=[128:16:352],
y=[96:16:288],par=[1.1 1.3]]

This is not written anywhere in the draft and I don't right now
know if this will hold in the general case, please
comment.

/Ingemar

From: Kyunghun Jung
[mailto:[email-masked]]
Sent: den 30 juli 2008
00:54
To: Even, Roni
Cc: Randell Jesup; Ingemar Johansson
S; [email-masked]; [email-masked]; [email-masked]
Subject: Re:
RE: [MMUSIC] [AVT] spatial-resolution parameter for RFC
3984bis

Thank
you Mr. Even Roni for the information.

We would like to ask experts in MMUSIC and AVT to pay a special
interest to the relationship

between imageattr and config/sprop-parameter-set. How to stack the set of
information in the SDP offer

will influence the session setup.

For example, suppose imageattr specifies image sizes over multiple
levels. The following form might be necessary.

m=video 49154 RTP/AVPF 99

b=AS:48

b=RS:0

b=RR:2500

a=rtpmap:99 MP4V-ES/90000

a=imageattr:99 [x=x1,y=y1] [x=x2,y=y2]
[x=x3,y=y3]

a=fmtp:99 profile-level-id=a; \
config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1

a=fmtp:99 profile-level-id=b; \

config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_2

a=fmtp:99 profile-level-id=c; \

config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_3

In case imageattr specifies image sizes within a single level.

a=imageattr:99 [x=x1,y=y1] [x=x2,y=y2]
[x=x3,y=y3]

a=fmtp:99 profile-level-id=8

a=fmtp:99 config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1

a=fmtp:99 config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_2

a=fmtp:99 config=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_3

The above forms are only basic possibilities and more
systematic design and suggestion are

requested from MMUSIC and AVT experts.

Have a nice meeting at Dublin!

Kyunghun Jung

_______________________________________________
mmusic mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/mmusic

---

From: Randell Jesup <[email-masked]>
Date: Thu, 31 Jul 2008 09:49:54 -0400

Kyunghun Jung <[email-masked]> writes:
>allocate frame buffer memory.   For MPEG-4 Visual streams, the frame size
>shall be extracted from the "config" information in the SDP. For H.264
>(AVC) streams, the frame size shall be extracted from the
>sprop-parameters-sets information in the SDP. For H.263 streams, a PSS
>server shall include the "a=framesize" field at the media level for each
>stream in SDP, and a PSS client should interpret this field, if present.
>Clients should be ready to receive SDP descriptions without this
>attribute.

a=framesize is not defined (that I know of) for H.263, nor is it generic.
It appears to be specific to 3GPP and not defined by any IETF RFC or draft.
The paragraph above doesn't specify the interactions between a=framesize
and H.263 fmtp lines ("CIF=2", etc).  Does a=framesize override the fmtp
preferences?  Must the value in a=framesize be listed in the fmtp, and what
happens if it isn't, or is that invalid?

>We might need to check whether other bitstream of config or
>sprop-parameter-sets does not change after the size information is
>removed,

I don't think you can remove the size fields from parameter sets, but I'd
have to check the spec to be sure.

-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS team
[email-masked]
"The fetters imposed on liberty at home have ever been forged out of the weapons
provided for defence against real, pretended, or imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)