Subject: [MMUSIC] Offer/Answer model does not work for H.264 parameters

From: Magnus Westerlund <[email-masked]>
Date: Thu, 25 Mar 2004 10:56:45 +0100

Hi,

In the effort to finalize H.264 we authors very trying to fine tune the 
Offer/Answer section the RTP payload format for H.264. Latest published 
version is:

http://www.ietf.org/internet-drafts/draft-ietf-avt-rtp-h264-04.txt

When doing this we have found a serious problem with the H.264 
parameters and the negotiation model in offer/answer (O/A). The problem 
is that H.264 has 5 parameters that are declarative and set by the media 
sender. These parameters are:
- max-don-diff
- init-buf-time
- parameter-sets
- interleaving-depth
- deint-buf-size

More information on them can be found in the last version of the RTP 
payload format. Some of them can also be used for capability 
declaration, but for that functionality there is no problem. The problem 
is when they are used to describe a media stream that will be sent to 
the receiver. For example parameter-sets is parameter carrying decoder 
configuration information. This information is determined by the encoder 
implementation and configuration, prior or during encoding (in case of 
pre-encoded material). The whole H.264 coder is built on giving the 
encoder the freedom to set the parameter-sets, and it can freely 
determine these parameters within the limitation set by the profile and 
level in use.

This clashes with the O/A model that is based on that the receiver 
declares what it can receive. For a media encoder and its payload format 
it means that all media parameters that one specifies are either a fixed 
configuration on how the information I will receive must look like, or a 
capability declaration. An example of the first case is the AMR payload 
format, where a receiver would specify that the payload format 
configuration used are octet-aligned mode with interleaving. A case of 
the second is the H.264 "max-br" parameter which defines the maximum 
bit-rate that a receiver is capable of handling, thus puts restriction 
on what the sender is allowed to use.

In the case of the 5 parameters above they all work in the opposite 
direction. The sender determines and declares the configuration that it 
will use. As a complication the possible values for these parameters are 
restricted through receiver declared capabilities. In the case of the 
parameter-sets the limitation is based on video codec profile and level 
as declared by the receivers "profile-level-id" parameter. In the case 
of the other parameters they are restricted by the capability version of 
the two parameters "interleaving-depth", and "deint-buf-size".

To get this two work in a full fledged way, the negotiation taking place 
between a H.264 sender and receiver would be the following.
1. Receiver declares its capabilities to sender.
2. Sender declares its configuration that fits within the receivers 
capabilities.

However I don't see how this would be possible to get to work in O/A. 
Any suggestion how to get this to work is greatly appreciated.

One possible solution that will give partial functionality would be the 
following.

Offerer declares its own capabilities on what it will receive. At the 
same time it assumes symmetric configuration and declares the parameters 
it will use to send that falls within its own receiver capabilities. 
This will require splitting the "interleaving-depth", and 
"deint-buf-size" two different parameters, one for capabilities and one 
for declaration. The offerer will have to make a leap of faith as it 
must construct the sender configuration prior to knowing the receivers 
capabilities.

The answerer receiver the offer. If its capabilities matches those of 
the sender it can declare its receiving capabilities equal to those 
capabilities. Then it also declares its sending parameters that fits 
within the offerer's capabilities. If the sender does not have matching 
capabilities it must remove that media alternative. To avoid total 
rejection of a stream the offerer can include multiple payload types 
with different levels of capabilities.

When the offerer receives the answer it checks which of its offers the 
answerer accepted and what parameters the answerer will use to send.

This will put certain restriction on how to handle the payload types and 
their numbers. Also the risk of rejection will be high if the offerer 
and answerer are not well matched, or the offerer goes to considerable 
effort of constructing multiple well structured payload types.

Any more problems with the above proposal?

Cheers

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

This communication is confidential and intended solely for the addressee(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you believe this message has been sent to you in error, please notify the sender by replying to this transmission and delete the message without disclosing it. Thank you.

E-mail including attachments is susceptible to data corruption, interruption, unauthorized amendment, tampering and viruses, and we only send and receive e-mails on the basis that we are not liable for any such corruption, interception, amendment, tampering or viruses or any consequences thereof.

_______________________________________________
mmusic mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mmusic