Subject: [AVT] RFC3984bis - profile & level in offer/answer

From: <[email-masked]>
Date: Wed, 10 Sep 2008 22:29:54 +0300

Folks,

I feel that this issue should be publicly discussed. 

The media type parameter profile-level-id includes three bytes,
profile_idc (8 bits), some flags and reserved bits (8 bits), and
level_idc (8 bits). It has been discussed that level 1b is signaled in
an ad-hoc manner, such that a lower value of level_idc does not always
mean a lower level. Actually, there is also an issue for profile, which
has not been mentioned as far as I know. 

For example, a bitstream with profile_idc=66 (Baseline profile) and
constraint_set1_flag (the second bit of the middle byte)=1 can also be
marked with profile_idc=77 (Main profile) and constraint_set0_flag (the
first flag)=1, and a Baseline profile decoder of a particular level can
decode any bitstream with constraint_set0_flag=1 with a level not higher
than the particular level. This means the following in SDP offer/answer:

- If the direction attribute is sendrecv, two different profile-level-id
values 42401E and 4D801E in the offer and the answer should be
considered equivalent. In this case, if other symmetric parameters
match, the offer/answer should be considered matched. Just to be more
complicated (but accurate :-), the second byte in the first value 42401E
can be replaced with any value with the second bit
(constraint_set1_flag) equal to 1, and the second byte in the second
value 4D801E can be replaced with any value with the first bit
(constraint_set0_flag) equal to 1, and the offer/answer are still
matching. 

- If the offer has direction attribute sendonly and
profile-level-id=42XX1E (XX is any value in the range of 00-FF,
inclusive), and the answer has direction attribute recvonly and
profile-level-id=YYZZ1E (YY is any value in the range of 00-FF,
inclusive, and ZZ is any value with the first bit equal to 1), then the
offer/answer should be considered matched.

And there are many other different combinations of profiles and
constraint flags to be considered to make the list complete. RFC 3984
considers all these cases as un-matched. 

So, opinion requested: should this issue be addressed in the bis draft?

BR, YK

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt

---

From: "Roni Even" <[email-masked]>
Date: Wed, 10 Sep 2008 22:54:25 +0300

YK,
My preference is to mandate profile match and level can be downgraded based
on the hierarchy.
I assume that 1b should be in the hierarchy between level 1 and 1.1
As for the flags my view is that they should be matched.

I think that H.264 compliance point is based on defined profiles. And the
levels gives hierarchy inside a specific profile.
I think that will make it simple enough. Note that there are more H264
parameters defined and offer enough challenge. 
Roni Even

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
[email-masked]
Sent: Wednesday, September 10, 2008 10:30 PM
To: [email-masked]
Subject: [AVT] RFC3984bis - profile & level in offer/answer

Folks,

BR, YK

---

From: <[email-masked]>
Date: Thu, 11 Sep 2008 07:53:34 +0300

Roni,

I don't have a strong opinion here. Just to point out that requiring the
flags to be matched (except for level 1b) has no problem for live
encoding, but for pre-encoded contents such a requirement may
essentially reduce interoperability. For example, a Baseline encoder
when it wants to generate a stream that is also decodable by a Main
decoder, it typically mark the bitstream with Baseline and set flag1,
while a Main encoder when it wants to generate a stream that is also
decodable by a Baseline decoder, it typically mark the bitstream with
Main and set flag0. 

BR, YK 

>From: ext Roni Even [mailto:[email-masked]] 
>Sent: 10 September, 2008 22:54
>To: Wang Ye-Kui (Nokia-NRC/Tampere); [email-masked]
>Subject: RE: [AVT] RFC3984bis - profile & level in offer/answer
>
>YK,

---

From: <[email-masked]>
Date: Thu, 11 Sep 2008 08:19:22 +0300

Note also that this issues applies equally to SVC and MVC, both are
extensions to H.264/AVC. 

BR, YK

>From: [email-masked] [mailto:[email-masked]] On 
>Behalf Of ext [email-masked]
>Sent: 11 September, 2008 07:54
>To: [email-masked]; [email-masked]
>Subject: Re: [AVT] RFC3984bis - profile & level in offer/answer
>
>
>Roni,
>

---

From: "Roni Even" <[email-masked]>
Date: Thu, 11 Sep 2008 16:35:13 +0300

YK,
The sender can offer both Main and Baseline. The text may clarify the issue
of the flags letting the receiver decide how to respond to the offer.
Roni 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Thursday, September 11, 2008 7:54 AM
To: [email-masked]; [email-masked]
Subject: RE: [AVT] RFC3984bis - profile & level in offer/answer

Roni,

BR, YK

---

From: <[email-masked]>
Date: Thu, 11 Sep 2008 23:28:40 +0300

Roni,

Note that H.264/AVC has MANY profiles, even not considering the SVC and
MVC ones. If the sender offers all, then you are going to have big SDP
messages... 

BR, YK 

>Sent: 11 September, 2008 16:35

---

From: "Roni Even" <[email-masked]>
Date: Fri, 12 Sep 2008 00:33:07 +0300

YK,
The definition of too many profiles, like you claim, is a JVT problem and
should not be addressed in AVT. My poor understanding is that a decoder
compatibility is based on profiles. In AVT we just specify how to negotiate
them we do not define new operation points or profiles here.

In your example you suggest that an offer has baseline profile and set flag1
the answer that can receive of baseline profile with no flags should be
considered as a common mode.
If this can be specified as a general rule like the flags can be ignored for
a common mode that is fine but if you want to specify that main profile with
flags x is equal to baseline profile with flags y and so on, I think that it
is not a good direction in term of complexity and you would be better with
offering both options.  

Roni

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Thursday, September 11, 2008 11:29 PM
To: [email-masked]; [email-masked]
Subject: RE: [AVT] RFC3984bis - profile & level in offer/answer

Roni,

BR, YK

---

From: <[email-masked]>
Date: Fri, 12 Sep 2008 09:28:51 +0300

Roni,

Offering both options when considering two profiles means offering N
options when considering N profiles. For the same example, if the
offerer guesses that the answerer may have a Baseline, Main or Extended
profile decoder, then three options need to be offered, Baseline with
flag1 set, Main with flag0 set, and Extended with both flag0 and flag1
set. And so on.

However, the burden of checking the complex profile equivalence
conditions can be moved to the answerer side - there can still be only
one option offerered and the spec can still be left as it is (or with
some informative text) but the answerer accepts the offer even its
decoder is of a different profile as long as the bitstream can be
decoded. To be able to make a communication in such scenarios, the
implementation of the offer/answer part needs to be complex anyway,
regardless whether we make this a normative part of the payload spec. 

The same analysis applies to declarative usages (e.g. the one-way thing
in unicast streaming or ATSC as Art mentioned). In my opinion, the key
pros and cons of the informative and the normative approaches are as
follows:

The informative approach makes the bis draft (and the SVC and MVC
payload drafts) more compatible to RFC 3984, but it leaves higher
opportunities for divergence in service and application standards (e.g.
ATSC, DVB, 3GPP), as then those bodies may specify something
differently, as Art pointed out. The normative way makes the bis draft
(and the SVC and MVC payload drafts) less compatible to RFC 3984, but it
will then basically be a common point for all service and application
standards. 

BR, YK 

>Sent: 12 September, 2008 00:33

---

From: "Roni Even" <[email-masked]>
Date: Fri, 12 Sep 2008 10:42:50 +0300

YK,
I am not objecting it is just from my humble experience, in interworking
between video offeres and answers, is that it is better to the offerer to
offer the options than counting on the answerer to understand the
equivalence unless the equivalence rules are simple. This allows the offerer
to be sure that the answer will be able to chose one mode.
If the rules are simple this is OK, otherwise if you want to define new
modes like saying Main with flags is Baseline, and give such a list I
believe that will lead to interoperability problems
Roni

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Friday, September 12, 2008 9:29 AM
To: [email-masked]; [email-masked]
Subject: RE: [AVT] RFC3984bis - profile & level in offer/answer

Roni,

BR, YK

---

From: <[email-masked]>
Date: Fri, 12 Sep 2008 13:10:10 +0300

Roni, All,

>in interworking
>equivalence unless the equivalence rules are simple. 

Good to know. Was this established just as a convention or because of
any technical reasons?

Anyway, we need to have a clear direction to go to proceed the bis
draft. So, please let me know which of the following 7 options we should
go ahead with this issue.
A) Do nothing.
B) Do something normative: B1) Offerer to offer multiple options
considering the possible equivalent values of profile-level-id; B2)
Answerer to consider the possible equivalent values of profile-level-id;
B3) A combination of B1 and B2.
C) Do something informative: C1) Offerer to offer multiple options
considering the possible equivalent values of profile-level-id; C2)
Answerer to consider the possible equivalent values of profile-level-id;
C3) A combination of B1 and B2.

BR, YK 

>Sent: 12 September, 2008 10:43
>Behalf Of

---

From: "Roni Even" <[email-masked]>
Date: Fri, 12 Sep 2008 18:45:51 +0300

YK,
This is not a convention it is just giving my experience from dealing with
interoperability between products. My moto is that if you want to make an
offer that will be understood use a simple solution. Assume that the remote
understands the simple case and may have problem understanding a complicated
logic in the standard.

As for which direction to go I suggest C3 which is C,C1 and C2
Give as an informative text about the relations between profile and flags.
(I am willing to have it as normative if the text is general, otherwise you
are defining profiles) and give informative text about offer and answer
behavior.

I am hesitant about normative text since I am not sure what this will do to
the H264 compliance. I am aware about the constrained baseline profile but
this is a special case that has its own profile_idc

BTW: If the offer is with Main profile and flags Should the response be Main
or can it be the baseline equivalent mode. And if the answer is the
equivalent baseline what should be the profile_idc in the encoded stream. 

Roni

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Friday, September 12, 2008 1:10 PM
To: [email-masked]; [email-masked]
Subject: RE: [AVT] RFC3984bis - profile & level in offer/answer

Roni, All,

BR, YK

---

From: <[email-masked]>
Date: Mon, 15 Sep 2008 09:17:55 +0300

Roni,

OK - so then let's go with C3 (which is a combination of C1 and C2) . 

Just not to let people be confused: the constrained baseline does not
have its own profile_idc, it can be at least signaled by profile_idc=66
and constraint_set1_flag=1 or profile_idc=77 and constraint_set0_flag=1,
as it was clearly agreed that the signaling part remains unchanged as in
old versions of H.264/AVC. 

BR, YK

>Sent: 12 September, 2008 18:46
>Subject: RE: [AVT] RFC3984bis - profile & flags semantics 
>
>YK,

---

From: "Roni Even" <[email-masked]>
Date: Mon, 15 Sep 2008 10:44:10 +0300

YK,
If this is the case you want to cover I have no problem with stating that
this two specific mode are equivalent and are also known as "constrained
baseline" and put a reference to the section in H.264 where it is defined.
I was more worried about not having a general rule and starting to define
new modes in the 3984bis draft
Roni

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Monday, September 15, 2008 9:18 AM
To: [email-masked]; [email-masked]
Subject: RE: [AVT] RFC3984bis - profile & flags semantics 

Roni,

BR, YK