Subject: RE: [AVT] RFC 3984 offer/answer Question

From: "Cesar Trobianni" <[email-masked]>
Date: Tue, 14 Aug 2007 09:17:08 -0700

>The level is the highest level supported
so level 3 means that you also support level 1. This is specified in RFC 3984. 

>B can offer level 3. 
 But still a separate profile-level-id ftmp line per level should be acceptable (as worst-case scenario), right? 

----------------------------------------
From: "Even, Roni" <[email-masked]>
Sent: Tuesday, August 14, 2007 7:38 AM
To: "Yuval Nissan" <[email-masked]>, <[email-masked]>
Subject: RE: [AVT] RFC 3984 offer/answer Question 

Yuval, 

The level is the highest level supported
so level 3 means that you also support level 1. This is specified in RFC 3984. 

B can offer level 3. 

Roni Even 

----------------------------------------

From:  Yuval Nissan
[mailto:[email-masked]] 

Sent: Monday, August 13, 2007 7:35
PM

To: [email-masked]

Subject: [AVT] RFC 3984
offer/answer Question 

Hi, 

I would like to clarify the following scenario: 

A and B use the same H.264 profile and packetization mode
(this assumption is only for simplicity).  

A supports up to level 3.0, and B supports only level 1.0. 

A doesn't know the level supported by B, and it wants
to start the negotiation as offerer. 

My understanding is that A needs to send in the SDP new
attribute for each level from 1.0 to 3.0 in order to ensure interoperability. Is
this correct? 

Regards, 

Yuval 

  

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: "Even, Roni" <[email-masked]>
Date: Tue, 14 Aug 2007 19:24:25 +0300

No, 

It is not needed since it does not give more options to B. 

If A wants later to limit the level to 1 it can do a re-invite.

Roni Even

________________________________

From: Cesar Trobianni [mailto:[email-masked]] 
Sent: Tuesday, August 14, 2007 7:17 PM
To: Even, Roni; [email-masked]; [email-masked]
Subject: RE: [AVT] RFC 3984 offer/answer Question

>The level is the highest level supported so level 3 means that you also support level 1. This is specified in RFC 3984.

________________________________

Yuval,

The level is the highest level supported so level 3 means that you also support level 1. This is specified in RFC 3984.

B can offer level 3.

Roni Even

________________________________

From: Yuval Nissan [mailto:[email-masked]] 
Sent: Monday, August 13, 2007 7:35 PM
To: [email-masked]
Subject: [AVT] RFC 3984 offer/answer Question

Hi,

A and B use the same H.264 profile and packetization mode (this assumption is only for simplicity). 

A doesn't know the level supported by B, and it wants to start the negotiation as offerer.

My understanding is that A needs to send in the SDP new attribute for each level from 1.0 to 3.0 in order to ensure interoperability. Is this correct?

Regards,

Yuval

---

From: Randell Jesup <[email-masked]>
Date: Tue, 14 Aug 2007 12:25:10 -0400

"Cesar Trobianni" <[email-masked]> writes:
>>The level is the highest level supported so level 3 means that you also
>>support level 1. This is specified in RFC 3984.
>>
>>B can offer level 3.
>
>But still a separate profile-level-id ftmp line per level should be
>acceptable (as worst-case scenario), right?

You can only have one fmtp line per payload value.

You can define multiple payloads with different fmtp's for a media stream,
however.  Responses (payloads and their fmtp's) are matched up against the
offer using a combination of profile-level-id, packetization-mode, and (if
used) the sprop interleave (if I remember correctly - that last one only
applies to packetization-mode 2).

Assuming anyone implements all of that totally correctly... :-/

-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS team
[email-masked]
"The fetters imposed on liberty at home have ever been forged out of the weapons
provided for defence against real, pretended, or imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)

---

From: "Yuval Nissan" <[email-masked]>
Date: Wed, 15 Aug 2007 09:18:46 +0300

I now understand that the following scenario is legal:

Offerer support up to level 3.
Offerer -> Answerer SDP message:

      m=video 49170 RTP/AVP 98
      a=rtpmap:98 H264/90000
      a=fmtp:98 profile-level-id=42A01E; packetization-mode=0;
                
Answerer support level 1 only.
Answerer -> Offerer SDP message:

     m=video 49170 RTP/AVP 98
     a=rtpmap:97 H264/90000
     a=fmtp:97 profile-level-id=42A00A; packetization-mode=0;
               
The offerer understands that the session should be in level 1.

Thanks for your feedback,
Yuval

-----Original Message-----
From: Randell Jesup [mailto:[email-masked]] 
Sent: Tuesday, August 14, 2007 19:25
To: [email-masked]
Cc: Yuval Nissan; [email-masked]; [email-masked]
Subject: Re: [AVT] RFC 3984 offer/answer Question

You can define multiple payloads with different fmtp's for a media
stream,
however.  Responses (payloads and their fmtp's) are matched up against
the
offer using a combination of profile-level-id, packetization-mode, and
(if
used) the sprop interleave (if I remember correctly - that last one only
applies to packetization-mode 2).

-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS
team
[email-masked]
"The fetters imposed on liberty at home have ever been forged out of the
weapons
provided for defence against real, pretended, or imaginary dangers from
abroad."
		- James Madison, 4th US president (1751-1836)

---

From: "Even, Roni" <[email-masked]>
Date: Wed, 15 Aug 2007 09:51:10 +0300

Yuval,

>      a=fmtp:97 profile-level-id=42A00A; packetization-mode=0;

You just have a problem with the payload type 98 

>      a=fmtp:98 profile-level-id=42A00A; packetization-mode=0;

> Sent: Wednesday, August 15, 2007 9:19 AM

> To: Randell Jesup; [email-masked]

> Cc: Even, Roni; [email-masked]

> >>The level is the highest level supported so level 3 means that you

> also

---

From: Randell Jesup <[email-masked]>
Date: Wed, 15 Aug 2007 07:40:33 -0400

>>      a=fmtp:98 profile-level-id=42A00A; packetization-mode=0;

Roni -

I think that's invalid for offer/answer, per 8.2.2.  If the offerer wants
to support only level 1.0 (in this case, 42a00a), it *has* to offer it
as a separate payload type in the offer.  The answer cannot add it as
the same payload type, and you're not supposed to add media formats
(payloads) in an answer.

8.2.2.  Usage with the SDP Offer/Answer Model

   When H.264 is offered over RTP using SDP in an Offer/Answer model [7]
   for negotiation for unicast usage, the following limitations and
   rules apply:

   o  The parameters identifying a media format configuration for H.264
      are "profile-level-id", "packetization-mode", and, if required by
      "packetization-mode", "sprop-deint-buf-req".  These three
      parameters MUST be used symmetrically; i.e., the answerer MUST
      either maintain all configuration parameters or remove the media
      format (payload type) completely, if one or more of the parameter
      values are not supported.

         Informative note: The requirement for symmetric use applies
         only for the above three parameters and not for the other
         stream properties and capability parameters.

---

From: "Even, Roni" <[email-masked]>
Date: Wed, 15 Aug 2007 15:16:49 +0300

Randell,

"If the offerer wants to support only level 1.0" - this is not the case
here. The offerer can support level 3. The answerer can support only
level 1 which is implied by the offer. The parameter value is supported.
This is why the answerer maintains the configuration parameters. 

Roni

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
A consensus means that everyone agrees to say collectively what no one
believes individually
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 

> Sent: Wednesday, August 15, 2007 2:41 PM
> To: Even, Roni
> Cc: Yuval Nissan; [email-masked]; [email-masked]
> >> Randell Jesup, Worldgate (developers of the Ojo videophone),
ex-Amiga
> >OS
> >
> >> team
> >
> >> [email-masked]
> >
> >> "The fetters imposed on liberty at home have ever been forged out
of

---

From: Randell Jesup <[email-masked]>
Date: Wed, 15 Aug 2007 13:58:11 -0400

>This is why the answerer maintains the configuration parameters. 

My apologies if I wrote too quickly - I was on the way to pick up my
mother-in-law at the hospital (after 10 weeks).  I should have said
"if the offerer wants to also support level 1.0".  It should have
been clear from the context, but I mis-stated it.

The offerer supports Level 3.0, and in the original question, only includes
a single payload and FMTP at 3.0.  According to 8.2.2 (below), you MUST use
profile-level-id symmetrically (along with packetization-mode and maybe
sprop-deint-buf-req).  The answer MUST not change any of them, or it must
remove the payload if it cannot support them.  So, when offered level 3, an
answerer who supports level 1 MUST remove the level 3 payload.

Accordingly (and per the much longer full section 8 of the RFC and RFC
3264, I think if you want to support levels 1-N you must offer levels 1-N
as separate payloads (N payloads).  If you want to support
packetization-modes 0 and 1 as well, you'll need 2*N payloads.

I wish I'd been looking at this before it was approved, though given
general offer-answer semantics it's not too surprising they chose this way.

>> I think that's invalid for offer/answer, per 8.2.2.  If the offerer
>> wants to support only level 1.0 (in this case, 42a00a), it *has* to
>> offer it as a separate payload type in the offer.  The answer cannot add
>> it as the same payload type, and you're not supposed to add media
>> formats (payloads) in an answer.
>> 
>>

---

From: "Even, Roni" <[email-masked]>
Date: Thu, 16 Aug 2007 09:35:47 +0300

Randell,
I can see your point but I would like to see other opinions maybe from
the authors of the text.

My view is that most current implementation of offer / answer for
unicast has the same approach. (this is based on interoperability tests
between vendors of video conferencing systems)

The problem here is that the profile-level-id parameter includes three
different parameters.
I still think that on the level part of the parameter the comparison
should be based on the highest common. I do not believe that the
intention here was to fix a specific mode.

Since the level parameter is an upper bound and lower levels are
included (except for level 1b) it does not make sense to offer a
separate media type for each level (there are 15 levels defined), for
example if the offerer support level 3 it will need to specify 3,2.2,
2.1, 2, 1.3, 1.2, 1.1 and 1 using 8 media type. 

I do not think that this was the intention in offer/answer. Offer/answer
should not be used to fix a specific level.
To make it more complicated, by using the other optional parameters like
max-mbps, max-fs the meaning of the level according to table A.1 in
H.264 is changed and the current maximum values supported are based on
the sum of the values from table A.1 and the optional parameters. 

BTW: I do not think that having the same level in the offer and answer
will make the communication symmetric, the encoder on each side can send
any frame size at any macro blocks per second rate that falls within the
specified level (this means frame size and frame rate would not be the
same on each direction). 
If you want to have a specific picture size or frame rate (e.g. CIF at
30 fps) then you will need a different signaling for requesting specific
mode. I mentioned in Chicago that we are working in H.323 on such
extensions (requesting a specific mode (picture size and frame rate))
and we will have a similar draft for SDP based systems.

Roni Even  

> Sent: Wednesday, August 15, 2007 8:58 PM
> To: Even, Roni
> Cc: Yuval Nissan; [email-masked]; [email-masked];
[email-masked]
> a single payload and FMTP at 3.0.  According to 8.2.2 (below), you
MUST
> use
> remove the payload if it cannot support them.  So, when offered level
3,
> an
> >> offer it as a separate payload type in the offer.  The answer
cannot
> add
> >>    When H.264 is offered over RTP using SDP in an Offer/Answer
model
> [7]

---

From: "Yuval Nissan" <[email-masked]>
Date: Thu, 16 Aug 2007 10:01:57 +0300

Roni,

Regarding the communication symmetric - if we understand it the way
which you propose (which I think is the practical one):

      m=video 49170 RTP/AVP 98
      a=rtpmap:98 H264/90000

     m=video 49170 RTP/AVP 98
     a=rtpmap:98 H264/90000
               
There can be 2 possibilities:

1. The offerer must transmit in level 1.0. Still the answerer can
transmit up to level 3.0 if he wants. 

2. Both of them must use level 1.

Which one is correct?
 
Yuval  

-----Original Message-----
From: Even, Roni [mailto:[email-masked]] 
Sent: Thursday, August 16, 2007 09:36
To: Randell Jesup
Cc: Yuval Nissan; [email-masked]; [email-masked];
[email-masked]

Randell,

Roni Even  

[email-masked]
MUST
3,
cannot
model

---

From: Randell Jesup <[email-masked]>
Date: Thu, 16 Aug 2007 09:21:30 -0400

>2. Both of them must use level 1.

Correct?  Neither, I'm afraid.

Given the RFC, if someone sends a response that doesn't comply with the
RFC (i.e. a level 1.0 response to level 3.0 offer), there's no way to know
how the offerer will deal with that response.

They could:
a) go asymmetric, and send 1.0 and receive any 1.0 -> 3.0 stream
b) require symmetric, and send and receive 1.0 only
c) Not understand the response and consider it in error, and either
   reINVITE (perhaps to 1.0, or to another codec, or even 3.0 again) or
   send a BYE.
d) Not understand the response and send 3.0 anyways (since the payload
   value was accepted, and the RFC doesn't allow for changing the level of
   a payload).
e) Process the response as a new, added payload (since it doesn't match
   on the triple of level, packetization, deint-buf) even though
   numerically it's the same.  Given offer-answer semantics, this would
   mean the answer did not provide a payload that was offered, and while 
   each side should continue to accept what they sent, the offerer
   should not send.  This is probably the most canonically-correct way to
   handle the response per the RFCs (3984 and 3264).

You don't know which of these they will do, especially if the
implementation was developed in a closed system without direct
interoperation tests (against systems that may not comply with the RFC and
so force them to do something other than option e, or whatever possible
mis-reading of the RFC they had).

Now, Roni may be correct that some of the conferencing people have assumed
that you don't have to match profile-level-id (which is incorrect per the
RFC, but if they all test against each other may work in practice).  If
there is a big hole in the RFC that has caused everyone to make some
non-compliant assumption (like option 'a' above), then it should be 
made into a new RFC (3984bis).  They may find that the authors had a reason
for not allowing it, or perhaps there's a better solution to their problem.

So neither is 'correct'.  One of those (or one of a-e) may be compatible
with many or even most H.264+SIP+SDP devices -- but I doubt that either
choice will avoid compatibility problems with all such devices.
Unfortunately.

---

From: "Even, Roni" <[email-masked]>
Date: Thu, 16 Aug 2007 16:58:17 +0300

Randell,
I went back to the mailing list and found the attached email.
It is in line with my thought that the level can be downgraded
Roni

> Sent: Thursday, August 16, 2007 4:22 PM
> To: Yuval Nissan
> Cc: Even, Roni; [email-masked]; [email-masked];
[email-masked];
> Ilan Avner

Subject: [AVT] Clarification on Offer Answer usage of H.264 payload format (RFC 3984)
From: "Magnus Westerlund" <[email-masked]>
To: "IETF AVT WG" <[email-masked]>
Cc: <[email-masked]>, "Stephan Wenger (Nokia)" <[email-masked]>, "Dave Singer" <[email-masked]>
Date: Wed, 12 Oct 2005 15:58:05 +0300

Hi,

We authors was made aware of an issue in the Offer/Answer usage for the 
H.264 RTP payload format by Tang Yongliang.

The issue is that section 8.2.2 says:

And later

"  o  Parameters declaring a configuration point are not downgradable,
       with the exception of the level part of the "profile-level-id"
       parameter."

That is contradicting for the "profile-level-id" parameter. The 
intention of the authors was that the profile part (profile_idc and 
profile_iop) would be fixed but the level (level_idc) can be downgraded. 
That way you as an receiver state the profiles and highest level for 
that profile you are willing to accept. The answerer can then respond 
with the same profile but with a possibly lower level.

So for clarification: An answerer MUST only maintain the profile_idc and 
profile_iop bytes of the profile-level-id value and MAY downgrade the 
level part.

Please note: This may require the offering party to make a new offer to 
provide its "sprop" parameters correctly due to the reduction in level.

However without this clarification the only way of getting a successful 
offer/answer for H.264 when not fully aware of the counter-parts 
capabilities would be to write one payload type for each profile-level 
combination that one could downgrade to.

Any comments on this clarification?

Cheers

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

---

From: Randell Jesup <[email-masked]>
Date: Thu, 16 Aug 2007 10:09:12 -0400

>I can see your point but I would like to see other opinions maybe from

Absolutely.

>between vendors of video conferencing systems)

I suspect that all the implementations you've tested (conferencing systems)
have tested against each other, and they've ended up (perhaps without even 
realizing it) at a non-compliant implementation.  Or perhaps one or more
realized the RFC meant the SDP processing would be a royal pain, and so
decided to purposely implement something similar but not RFC compliant.

>intention here was to fix a specific mode.

Perhaps it wasn't, but that is the result of the text (or more to the
point, fix a 1<->1 relationship of payload value to level).  You can
support a lot of levels, but the RFC says you have to be able to
distinguish them by payload value.  It ends up wordy in the SDP (VERY
wordy), but it would work.  (It would make support for 3984 above a low
level pretty much impossible on UDP SIP, for example, but it's not the only
thing that can do this - see Magnus's 10K SDP example.)

>2.1, 2, 1.3, 1.2, 1.1 and 1 using 8 media type. 

Lower levels are included, but from the registration for profile-level-id
in 8.1:

                        If the profile-level-id parameter is used to
                        indicate properties of a NAL unit stream, it
                        indicates the profile and level that a decoder
                        has to support in order to comply with [1] when
                        it decodes the stream.  

(that is declaritive SDP) and for negotiation:

                        If the profile-level-id parameter is used for
                        capability exchange or session setup procedure,
                        it indicates the profile that the codec
                        supports and the highest level
                        supported for the signaled profile.  The
                        profile-iop byte indicates whether the codec
                        has additional limitations whereby only the
                        common subset of the algorithmic features and
                        limitations of the profiles signaled with the
                        profile-iop byte and of the profile indicated
                        by profile_idc is supported by the codec.  For
                        example, if a codec supports only the common
                        subset of the coding tools of the Baseline
                        profile and the Main profile at level 2.1 and
                        below, the profile-level-id becomes 42E015, in
                        which 42 stands for the Baseline profile, E0
                        indicates that only the common subset for all
                        profiles is supported, and 15 indicates level
                        2.1.

                            Informative note: Capability exchange and
                            session setup procedures should provide
                            means to list the capabilities for each
                            supported codec profile separately.  For
                            example, the one-of-N codec selection
                            procedure of the SDP Offer/Answer model can
                            be used (section 10.2 of [7]).

10.2 of RFC 3264 is the "offer inactive with many payloads, get inactive
response, reINVITE active with a single payload, get active response".

That (and the "list the capabilities for each supported codec profile
separately") makes me think that this requirement to have N (or 2*N or
in theory even more) payloads is intentional, not just poor wording.

>should not be used to fix a specific level.

See above; I'm not sure you're right here.  I agree that (unless there's
something I'm missing about why they did this) I'd really like to be
able to have an asymmetric response, but there's no standard (see my
options a-e in my response to Yuval).

>the sum of the values from table A.1 and the optional parameters. 

Right, though the offer-answer section says that those do not have to
match in the offer and answer.

>specified level (this means frame size and frame rate would not be the

You're correct it doesn't select actual frame size (in x, y) or frame rate.
(within the limits of the level).  However, the assumption of the writers
*may* have been that agreeing on a level was important due to some
algorithmic differences or resource-reservation issues - for example, an
offerer who does a lot of streams might not want to have to reserve enough
processing power, buffers, etc to decode level 3 if the answerer only
supports level 1 - but if you let the answerer respond at level 1 to a
level 3 offer, the offerer doesn't know if they'll receive level 1 or 3.
(That's totally a guess and likely wrong, but it's an example of why they
might have made the decision.)

>and we will have a similar draft for SDP based systems.

Which will be useful when it's available.  Our device in reality only
supports fixed resolutions.  It can support others, but it has to scale
them to fit on the display, so it's better to avoid that, and scaling isn't
free (horsepower-wise, or artifact-wise).

We may need a 3984-bis, or an explanation why all the conferencing people
are doing it wrong or have mis-read the RFC (it's dense reading - witness
this discussion.  And some people read it and say "that's silly, they must
have meant to do it this way..." since they don't know why the rfc reads
the way it does - me included.)

---

From: "Even, Roni" <[email-masked]>
Date: Thu, 16 Aug 2007 17:20:49 +0300

Randell
The text in RFC 3984

Parameters declaring a configuration point are not downgradable,
      with the exception of the level part of the "profile-level-id"
      parameter.  This expresses values a receiver expects to be used
      and must be used verbatim on the sender side.

Explains that you can downgrade the level parameter

Roni

> Sent: Thursday, August 16, 2007 5:09 PM
> To: Even, Roni
[email-masked]

---

From: Randell Jesup <[email-masked]>
Date: Thu, 16 Aug 2007 10:21:05 -0400

>It is in line with my thought that the level can be downgraded

Darn! :-)  I knew this whole conversation sounded familiar.  I remember
(now) seeing that, and in fact I've kept the article "ticked" in Gnus
so it doesn't disappear on me.

There really should be an rfc 3984bis (or at least an informational RFC; I
don't know how this is supposed to work); everyone who comes to read it
without searching the mail-archives will make the same set of mistakes and
mis-readings.

Magnus?

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 16 Aug 2007 08:43:43 -0700

3984 bis is on our list for a while now.  It has almost become a  
ritual between (at least) Magnus and me: a the end of an AVT session,  
we talk and say "we really should get started on 384 bis".
Seriously, this is not the only known ambiguity in the spec.   
Somewhere, I even have a list of the others :-).  I'll devote cycles  
once the ccm spec is out (hint hint).
Stephan

On Aug 16, 2007, at 7:21 AM, Randell Jesup wrote:

> of the weapons

---

From: "Tom Kristensen" <[email-masked]>
Date: Thu, 23 Aug 2007 13:29:05 +0200

> without searching the mail-archives will make the same set of mistakes and

Agree!

However, reading the RFC3984 carefully reveals the "hidden statement"
on level part being downgradable.

-- Tom

-- 
# TANDBERG R&D
## http://www.tandberg.com
### http://folk.uio.no/tomkri/