Subject: [AVT] Determining the desired transmitted resolution and frame rate when using H.264

From: Shahaf Rozanski <[email-masked]>
Date: Tue, 3 Jul 2007 11:16:47 +0300

Hi,

In the SDP negotiation each agent declares it supported receive capabilities. For H.264 it can be done using the profile-level-id mechanism suggested in RFC 3984.
Let's assume that the offerer declared baseline profile level 1.1. As a result he is willing to receive different configuration e.g. up to CIF resolution with 7.5 FPS or up to QCIF with 30 FPS.
Is there a method for the offerer to hint the answerer that he would like to receive in a specific configuration (e.g. QCIF 30FPS)?
This can be useful if the offerer is a UE that is best suited for QCIF resolutions.

Thanks,
Shahaf
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: Gunnar Hellström <[email-masked]>
Date: Tue, 3 Jul 2007 17:31:48 +0200

Shahaf,
This is an important question. 
Not only for the UA, but for the user.
For example a sign language user will experience CIF 7.5 fps totally
unusable, while QCIF 30 fps is quite good and conveys everything except the
most fine expressions of sign language.
 
So, it is very desirable that UA:s are configurable to express the
preference.
 
I rapidly checked RFC 4629 for H.263, and found in section 8.2.1 about the
Offer-answer model, the following sentence:
 
" 

   Parameters offered first are the most preferred picture mode to be
   received.
"

Please check RFC 3984 for a similar statement. I guess it is there.

/Gunnar

-------------------------------------------------------------------
Gunnar Hellström
Omnitor
[email-masked]
  _____  

From: Shahaf Rozanski [mailto:[email-masked]] 
Sent: Tuesday, July 03, 2007 10:17 AM
To: [email-masked]
Cc: Yuval Nissan; Keren Malachi
Subject: [AVT] Determining the desired transmitted resolution and frame rate
when using H.264 

Hi,

In the SDP negotiation each agent declares it supported receive
capabilities. For H.264 it can be done using the profile-level-id mechanism
suggested in RFC 3984.

Let’s assume that the offerer declared baseline profile level 1.1. As a
result he is willing to receive different configuration e.g. up to CIF
resolution with 7.5 FPS or up to QCIF with 30 FPS.

Is there a method for the offerer to hint the answerer that he would like to
receive in a specific configuration (e.g. QCIF 30FPS)?

This can be useful if the offerer is a UE that is best suited for QCIF
resolutions. 

Thanks,

Shahaf

__________ NOD32 2372 (20070703) Information __________

Detta meddelande dr genomsvkt av NOD32 Antivirus.
http://www.nod32.com

---

From: "Even, Roni" <[email-masked]>
Date: Tue, 3 Jul 2007 18:55:42 +0300

Hi,

H.264 does not have the definition of fixed frame sizes. The level gives
some maximum values which enable you to estimate the maximum size of the
frame and the frame rate.

Using max-mbps, max-fs you can override the level parameters. Still this
does not allow you to request a specific resolution or number of macro
b;ocks per second.

There is currently work done at the ITU on defining additional
parameters to H.241 in order to allow such functionality and after it is
done there I assume we will try to update RFC 3984 with this
functionality.

BTW: there are some new parameters now in
draft-ietf-avt-rtp-h264-params-00 and you should look at it.

Roni Even

________________________________

From: Shahaf Rozanski [mailto:[email-masked]] 
Sent: Tuesday, July 03, 2007 11:17 AM
To: [email-masked]
Cc: Yuval Nissan; Keren Malachi
Subject: [AVT] Determining the desired transmitted resolution and frame
rate when using H.264 

Hi,

In the SDP negotiation each agent declares it supported receive
capabilities. For H.264 it can be done using the profile-level-id
mechanism suggested in RFC 3984.

Let's assume that the offerer declared baseline profile level 1.1. As a
result he is willing to receive different configuration e.g. up to CIF
resolution with 7.5 FPS or up to QCIF with 30 FPS.

Is there a method for the offerer to hint the answerer that he would
like to receive in a specific configuration (e.g. QCIF 30FPS)?

Thanks,

Shahaf

---

From: Randell Jesup <[email-masked]>
Date: Tue, 03 Jul 2007 14:03:30 -0400

>frame and the frame rate.

The problem (as Roni states) is that while the profile/etc may determine
the base ability to decode (horsepower, memory, etc), often the decoded
video is fed to devices that cannot render/display arbitrary resolutions
(and not just the aspect ratio issue which is currently under discussion).
In theory, a device can probably always display the video in some manner,
but it may not be a usable/useful manner (cropping or adding (large) areas
of black).  Not all devices will have an arbitrary scaler, or even a scaler
at all, or they may only have fixed scalers to choose from, or may not have
the horsepower to run a scaler while doing max-mbps encode/decode.

>does not allow you to request a specific resolution or number of macro

Right, not even to express a preference or tradeoff.  MAYBE you could do a
little of it using capability negotiation - if the other side understood
what you're trying to do.

>done there I assume we will try to update RFC 3984 with this

Where is this being done?  Wouldn't it make sense to add input from the
IETF side now, so that when everyone is done there's a reasonable mapping
back and forth?

-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS team
[email-masked]
"The fetters imposed on liberty at home have ever been forged out of the weapons
provided for defence against real, pretended, or imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)

---

From: Gunnar Hellström <[email-masked]>
Date: Wed, 4 Jul 2007 00:01:00 +0200

The order of the payload types in the m-line should reflect the order of
preference to receive the difference payload types.
 
Different H.264 parameter sets are combined under different payload types as
specified in RFC 4629.
 
This seems to me to offer a possibility to indicate your preferences.
 
Is it not right that you would indicate QCIF preference over CIF by
declaring the following sdp parameters:
 
 m=video 49170 RTP/AVP 100 101
     a=rtpmap:100 H264/90000
     a=fmtp:100 profile-level-id=42A00C;maxfs=99
     a=rtpmap:101 H264/90000
     a=fmtp:101 profile-level-id=42A00B;maxfs=396;maxmbps=2900

 
/Gunnar

  _____  

From: Even, Roni [mailto:[email-masked]] 
Sent: Tuesday, July 03, 2007 5:56 PM
To: Shahaf Rozanski; [email-masked]
Cc: Yuval Nissan; Keren Malachi
Subject: RE: [AVT] Determining the desired transmitted resolution and
framerate when using H.264 

Hi,

There is currently work done at the ITU on defining additional parameters to
H.241 in order to allow such functionality and after it is done there I
assume we will try to update RFC 3984 with this functionality.

BTW: there are some new parameters now in draft-ietf-avt-rtp-h264-params-00
and you should look at it.

Roni Even

  _____  

Hi,

Thanks,

Shahaf

---

From: Randell Jesup <[email-masked]>
Date: Tue, 03 Jul 2007 22:44:58 -0400

>     a=fmtp:101 profile-level-id=42A00B;maxfs=396;maxmbps=2900

Good point, though this is a form of capability negotation possible within
the original SDP/offer-answer framework.  Also, since H.264 fmtp lines can
get *long*, this might add to the fun for SIP over UDP.

That would prefer QCIF (or so, depending on aspect ratio) over CIF (or
so) - but they could accept 101 and send QCIF (or anything with <= 396
MB/frame).  Perhaps the aspect-ratio drafts would help here, but they can
still send smaller frames.

It's worth considering promoting this as a way to negotiate that, if there
aren't any "gotchas".  Note that some UAs might not handle this well, but
if they really implement all of the RFC it should.

---

From: Gunnar Hellström <[email-masked]>
Date: Wed, 4 Jul 2007 07:05:59 +0200

OK, so we found that it seems possible to express some order of preference
for video parameters for H.264 by the order of payload types in the m-line.

Next question is if it is possible to compose parameter collections that
correspond to what I want to express.

Shahaf may have had a slightly different need when starting this discussion.
I continue on my example with the hope that it has similarities.

Assume that I have a capable receiver and transmitter, that can receive and
transmit CIF 30 fps.
In calls with other capable UAs, I of course want to get CIF 30 fps.
But in calls with a weaker UA, I want to express preference for getting QCIF
30 rather than CIF 15.

Can I collect three sets of parameters in different PTs, expressing:

CIF - 30 , QCIF - 30, CIF - 15.

I am afraid that with my previous example, it might have been perceived as
if I had a total preference for QCIF (or so).

This is an important need from sign language communication. We need to
discourage frame rates lower than 20 fps. If this method is too clumsy, we
should start on some other method. 

/gunnar
 
--------------------------------------------------------

Gunnar Hellström <[email-masked]> writes:
>The order of the payload types in the m-line should reflect the order 
>of preference to receive the difference payload types.
> 
>Different H.264 parameter sets are combined under different payload 
>types as specified in RFC 4629.
> 

It's worth considering promoting this as a way to negotiate that, if there
aren't any "gotchas".  Note that some UAs might not handle this well, but if
they really implement all of the RFC it should.

--
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS
team [email-masked] "The fetters imposed on liberty at home have ever been
forged out of the weapons provided for defence against real, pretended, or
imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)

Detta meddelande är genomsökt av NOD32 Antivirus.
http://www.nod32.com

---

From: "Even, Roni" <[email-masked]>
Date: Wed, 4 Jul 2007 09:45:28 +0300

Randell,
The work is done in Q1 of study group 16. I will try to write a draft
explaining the problem and the suggested solution. It is still a work in
process with the intention to have a proposal in beginning of 2008.
Roni

> -----Original Message-----
> From: Randell Jesup [mailto:[email-masked]]
> Sent: Tuesday, July 03, 2007 9:04 PM
> To: Even, Roni
> Cc: Shahaf Rozanski; [email-masked]; Yuval Nissan; Keren Malachi
> Subject: Re: [AVT] Determining the desired transmitted resolution and
> frame rate when using H.264
> 
> team
> "The fetters imposed on liberty at home have ever been forged out of
the
> weapons
> provided for defence against real, pretended, or imaginary dangers
from
> abroad."

---

From: "Yuval Nissan" <[email-masked]>
Date: Wed, 4 Jul 2007 09:51:11 +0300

Roni,

I think the ability to negotiate desirable working point (inside the
envelope of the level) should be general method for all video coders and
not specific (and different today) for each one of them. Do you see any
technical reason why it can't be that way?

(Take for example transcoder server that need to support few video
coders...)

Regards,
Yuval    

-----Original Message-----
From: Even, Roni [mailto:[email-masked]] 
Sent: Wednesday, July 04, 2007 9:45 AM
To: Randell Jesup
Cc: Shahaf Rozanski; [email-masked]; Yuval Nissan; Keren Malachi
Subject: RE: [AVT] Determining the desired transmitted resolution and

Randell,

the
from

---

From: "Yuval Nissan" <[email-masked]>
Date: Wed, 4 Jul 2007 09:51:54 +0300

Hi,

Regards,

Yuval    

Sent: Tuesday, July 03, 2007 6:56 PM

Hi,

Roni Even

Hi,

Thanks,

Shahaf

---

From: "Even, Roni" <[email-masked]>
Date: Wed, 4 Jul 2007 09:54:00 +0300

Gunnar,
The frame rate is dependent on the content. If the input in a meeting room it may be better to favor high frame rate on resolution. On the other hand if you send a presentation (like H.239 content) it is better to have high resolution and low frame rate.
Roni 

> From: Gunnar Hellström [mailto:[email-masked]]
> Sent: Wednesday, July 04, 2007 8:06 AM
> To: 'Randell Jesup'
> Cc: 'Shahaf Rozanski'; Even, Roni; 'Yuval Nissan'; [email-masked]; 'Keren
> Malachi'
> Subject: RE: [AVT] Determining the desired transmitted resolution
> andframeratewhen using H.264
> 
> if

---

From: "Even, Roni" <[email-masked]>
Date: Wed, 4 Jul 2007 09:59:19 +0300

Yuval,
You can do it today in H.263 and H.261. The problem is that H.264 does
not define fixed resolutions so using the max-fs gives the number of
macro blocks and not the geometry. 
If you read the thread and saw Randell text you will notice that picture
aspect ratio is only part of the information needed in order to create a
picture that is not distorted. The video input today (camera) is
different than in the past, you have 4:3 or 16:9 cameras and they create
different geometry.
This is why I suggested you look also at the new h264 parameters draft.

Roni

> From: Yuval Nissan [mailto:[email-masked]]
> Sent: Wednesday, July 04, 2007 9:51 AM
> To: Even, Roni; Randell Jesup
> Cc: Shahaf Rozanski; [email-masked]; Keren Malachi
> > >some maximum values which enable you to estimate the maximum size
of
> > at all, or they may only have fixed scalers to choose from, or may
not
> > have
> > Right, not even to express a preference or tradeoff.  MAYBE you
could
> do a
> > >parameters to H.241 in order to allow such functionality and after
it
> is
> > Randell Jesup, Worldgate (developers of the Ojo videophone),
ex-Amiga
> OS