Subject: [AVT] [Fwd: WGLC on RFC3984bis and SVC]

From: Tom Taylor <[email-masked]>
Date: Mon, 12 Jan 2009 16:20:49 +0100

Here's a reminder of your homework over the holidays :)

-------- Original Message --------
Subject: [AVT] WGLC on RFC3984bis and SVC
Date: Tue, 16 Dec 2008 21:12:35 -0500
From: Tom Taylor <[email-masked]>
To: IETF AVT WG <[email-masked]>, Roni Even <[email-masked]>

I have consulted with Roni, who pointed out that other SDOs have a dependency on
the subject drafts. We are therefore announcing a Working Group Last Call on the
following documents:

draft-ietf-avt-rfc3984bis-02.txt, RTP Payload Format for H.264 Video

draft-ietf-avt-svc-16.txt, RTP Payload Format for SVC Video

Given the holidays, WGLC will last until January 16.

Tom Taylor
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt

---

From: "Jonathan Lennox" <[email-masked]>
Date: Thu, 15 Jan 2009 16:37:39 -0500

I've read over 3984bis -- just a few comments.  Generally this seems
good.

The Parameter Set Considerations section (8.4) should contain
recommendations on when a parameter set sent out-of-band should be
repeated in-band.  In particular, I'd suggest that source-specific
parameter sets sent using the fmtp source parameter SHOULD also be sent
in-band in case you're talking to a 3984-only endpoint, which won't
understand source parameters.  This should also be mentioned in Section
17 (3984 compatibility)'s discussion of item 7 in the changes from 3984.

Additionally, even for those combinations of parameter set transmission
mechanisms that are NOT RECOMMENDED, there needs to be guidance to a
receiver as to what it should do if it, e.g., receives different
parameter sets with the same ID both in-band and out-of-band.  Since
mixing in-band updates and out-of-band updates in a session is
forbidden, it looks like the receiver should assume that the out-of-band
parameter sets were pre-session, and the in-band ones supersede them.  I
think this should be explicitly spelled out, however.

On another point, if you're in a topology in which an RTP session
carries multiple independent sources, it's not clear whether the max-*
parameters (Section 8.1) apply to a receiver's capability for a single
source, or for its capability for all sources in the session.  Based on
the descriptions it looks like it's the former, since some of the
parameters are described in terms of the HRD and the like, and in
general this makes sense since in such topologies it's often impossible
for encoders to coordinate so as to stay within receivers' limits.  But
this should probably be made explicit.

Editorial:
Section 1.1, first paragraph: "H.264" is not an acronym.  Perhaps
rephrase as "In this memo the name H.264 is used".

Second paragraph: "all forms of digital compressed video from, low-bit
rate...": no comma.

-- 
Jonathan Lennox
Vidyo, Inc
[email-masked]

---

From: "Jonathan Lennox" <[email-masked]>
Date: Thu, 15 Jan 2009 18:02:28 -0500

I've read over the SVC draft.  Thanks to everyone for the good work!

Section 4.9 (PACSI), p. 38, first full paragraph following the figure:
As mentioned later, the bits S and E are defined if the bit Y is 1, not
if the bit X is 1.  I suggest the wording:

The bits A, P, and C are specified only if the bit X is equal to 1.  The
fields TL0PICIDX and IDRPICID are present, and the bits S and E are
specified, only if the bit Y is equal to 1.

Section 5.1 (Packetization rules for SST), p. 47, Paragraph 3
(Informative note):  This note should mention NI-MTAP as well.

Section 7.2.1 (Mapping of Payload Type Parameters to SDP):  I believe it
would be useful to be able to represent sprop-scalability-info and
sprop-operation-point-info in the fmtp source attribute as well.  I
don't currently see any motivation for including
sprop-no-NAL-reordering-required or sprop-avc-ready in the fmtp source
attribute, but they might be worth considering.

Editorial: Section 1.1.2, p. 7, third para, last line:
"conforming to one or more of the profile specified in"... should be
"profiles"

Section 4.3, p. 26, 2nd paragraph: "Fragment Unit" should be
"Fragmentation Unit".

-- 
Jonathan Lennox
Vidyo, Inc
[email-masked]

---

From: Ye-Kui Wang <[email-masked]>
Date: Fri, 16 Jan 2009 13:32:33 -0500

Hi Jonathan,

Thanks for reviewing the draft and your comments. 

> recommendations on when a parameter set sent out-of-band 
> should be repeated in-band.  

There are use cases to in-band transmit additional parameter sets and
updated parameter sets, but I don't see any good use case for repeating
in-band a parameter set that has been sent out-of-band, though that is not
disallowed. Therefore, maybe we don't need recommendations in 8.4 for
in-band repeating of parameter sets out-of-band transmitted?

> In particular, I'd suggest that 
> source-specific parameter sets sent using the fmtp source 
> parameter SHOULD also be sent in-band in case you're talking 
> to a 3984-only endpoint, which won't understand source 
> parameters.  

The following bullet item in section 8.2.2 prescribes a MUST for in-band
parameter set transport in such a situation. 
o	When there is a "sprop-parameter-sets" conveyed using the "fmtp"
source attribute as specified in section 6.3 of [9], and the answerer does
not understand the "fmtp" source attribute, in-band transport of parameter
sets MUST be used, and the answerer MUST NOT include
"use-level-src-parameter-sets" equal to 1 or the "fmtp" source attribute in
the answer.

Or do you think we should repeat those parameter set specifications from
section 8.2.2 in section 8.4?

> This should also be mentioned in Section

The text was drafted considering the above MUST for in-band parameter set
transport. With this in mind, do you still think that something more need to
be mentioned in section 17?

> Additionally, even for those combinations of parameter set 
> transmission mechanisms that are NOT RECOMMENDED, there needs 
> to be guidance to a receiver as to what it should do if it, 
> e.g., receives different parameter sets with the same ID both 
> in-band and out-of-band.  Since mixing in-band updates and 
> out-of-band updates in a session is forbidden, it looks like 
> the receiver should assume that the out-of-band parameter 
> sets were pre-session, and the in-band ones supersede them.  
> I think this should be explicitly spelled out, however.
> 

Which parameter set takes effect is prescribed by the H.264/AVC
specification. Assuming there are one out-of-band picture parameter set A
and one in-band picture parameter set B, both transported during a session,
and share the same ID. If A and B are assoicated with different access
units, then the one associated with the access unit earlier in decoding
order supersedes the other, at and after the access unit. If they are
assoicated with the same access unit, their content must the same according
to the following restriction to picture parameter sets copied from section
7.4.1.2.1 of the H.264/AVC specification. 

	Any picture parameter set NAL unit containing the value of
pic_parameter_set_id for the active picture parameter set RBSP for a coded
picture shall have the same content as that of the active picture parameter
set RBSP for the coded picture unless it follows the last VCL NAL unit of
the coded picture and precedes the first VCL NAL unit of another coded
picture.

Therefore, I personally don't see a problem. However, should we somehow
repeat the above in section 8.4? 

> On another point, if you're in a topology in which an RTP 
> session carries multiple independent sources, it's not clear 
> whether the max-* parameters (Section 8.1) apply to a 
> receiver's capability for a single source, or for its 
> capability for all sources in the session.  Based on the 
> descriptions it looks like it's the former, since some of the 
> parameters are described in terms of the HRD and the like, 
> and in general this makes sense since in such topologies it's 
> often impossible for encoders to coordinate so as to stay 
> within receivers' limits.  But this should probably be made explicit.
> 

I agree. However, when one source is something like an MCU, behind which
there are more sources, then the capability parameter sets assoicated with
the MCU source should be intepreted as the minimum set of the union of the
MCU source and the sources behind it. I will make a change for this in the
next version. Let me know if you have a different opinion.

> Editorial:
> Section 1.1, first paragraph: "H.264" is not an acronym.  
> Perhaps rephrase as "In this memo the name H.264 is used".
> 

Good point. Corrected.

> Second paragraph: "all forms of digital compressed video from, low-bit

I moved the comman before "from".

BR, YK

---

From: Ye-Kui Wang <[email-masked]>
Date: Fri, 16 Jan 2009 13:42:42 -0500

A few complementary points, see inline below. 

BR, YK

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Ye-Kui Wang
> Sent: Friday, January 16, 2009 1:33 PM
> To: 'Jonathan Lennox'; 'IETF AVT WG'
> Subject: Re: [AVT] WGLC on RFC3984bis: comments
> 
> Hi Jonathan,
> 
> > repeated in-band.
> There are use cases to in-band transmit additional parameter 
> sets and updated parameter sets, but I don't see any good use 
> case for repeating in-band a parameter set that has been sent 
> out-of-band, though that is not disallowed. Therefore, maybe 
> we don't need recommendations in 8.4 for in-band repeating of 
> parameter sets out-of-band transmitted?
> 
> > source-specific parameter sets sent using the fmtp source parameter 
> > SHOULD also be sent in-band in case you're talking to a 3984-only 
> > endpoint, which won't understand source parameters.
> 
> The following bullet item in section 8.2.2 prescribes a MUST 
> for in-band parameter set transport in such a situation. 
> source attribute as specified in section 6.3 of [9], and the 
> answerer does not understand the "fmtp" source attribute, 
> in-band transport of parameter sets MUST be used, and the 
> answerer MUST NOT include "use-level-src-parameter-sets" 
> equal to 1 or the "fmtp" source attribute in the answer.
> 
> Or do you think we should repeat those parameter set 
> specifications from section 8.2.2 in section 8.4?
> 
> The text was drafted considering the above MUST for in-band 
> parameter set transport. With this in mind, do you still 
> think that something more need to be mentioned in section 17?
> 
> > transmission mechanisms that are NOT RECOMMENDED, there needs to be 
> > guidance to a receiver as to what it should do if it, e.g., 
> receives 
> > different parameter sets with the same ID both in-band and 
> > out-of-band.  Since mixing in-band updates and out-of-band 
> updates in 
> > a session is forbidden, it looks like the receiver should 
> assume that 
> > the out-of-band parameter sets were pre-session, and the 
> in-band ones 
> > supersede them.
> Which parameter set takes effect is prescribed by the 
> H.264/AVC specification. Assuming there are one out-of-band 
> picture parameter set A and one in-band picture parameter set 
> B, both transported during a session, and share the same ID. 
> If A and B are assoicated with different access units, then 
> the one associated with the access unit earlier in decoding 
> order supersedes the other, at and after the access unit. If 
> they are assoicated with the same access unit, their content 
> must the same according to the following restriction to 
> picture parameter sets copied from section
> 	Any picture parameter set NAL unit containing the value 
> of pic_parameter_set_id for the active picture parameter set 
> RBSP for a coded picture shall have the same content as that 
> of the active picture parameter set RBSP for the coded 
> picture unless it follows the last VCL NAL unit of the coded 
> picture and precedes the first VCL NAL unit of another coded picture.
> 

The situation is the same for sequence parameter sets.

> Therefore, I personally don't see a problem. However, should 
> we somehow repeat the above in section 8.4? 
> 
> > source, or for its capability for all sources in the 
> session.  Based 
> > on the descriptions it looks like it's the former, since 
> some of the 
> > general this makes sense since in such topologies it's often 
> > impossible for encoders to coordinate so as to stay within 
> receivers' 
> > limits.  But this should probably be made explicit.
> > 
> 
> I agree. However, when one source is something like an MCU, 
> behind which there are more sources, then the capability 
> parameter sets assoicated with the MCU source should be 
> intepreted as the minimum set of the union of the MCU source 
> and the sources behind it. I will make a change for this in 
> the next version. Let me know if you have a different opinion.
> 

This applies to any media type parameter when it is used to indicate a
receiver capability.

---

From: "Roni Even" <[email-masked]>
Date: Fri, 16 Jan 2009 20:56:21 +0200

YK,

Roni - I think that the IDR procedure described in the draft is such a case.

---

From: Ye-Kui Wang <[email-masked]>
Date: Fri, 16 Jan 2009 14:09:08 -0500

Roni,

Good point. Any other use case? What kind of recommendation should we have
for in-band repeating of parameter sets already out-of-band transmitted?

BR, YK 

> From: Roni Even [mailto:[email-masked]] 
> Sent: Friday, January 16, 2009 1:56 PM
> To: 'Ye-Kui Wang'; 'Jonathan Lennox'; 'IETF AVT WG'
> Subject: RE: [AVT] WGLC on RFC3984bis: comments
> 
> YK,
>

---

From: "Roni Even" <[email-masked]>
Date: Fri, 16 Jan 2009 21:15:07 +0200

YK,
My understanding is that the last one you get is the one you use. The only
recommendation we can make is that this may happen and therefore we have
already some text saying that when a sender assigns ID it should try to use
different ones.

Roni 

-----Original Message-----
From: Ye-Kui Wang [mailto:[email-masked]] 
Sent: Friday, January 16, 2009 9:09 PM
To: 'Roni Even'; 'Jonathan Lennox'; 'IETF AVT WG'
Subject: RE: [AVT] WGLC on RFC3984bis: comments

Roni,

BR, YK

---

From: stephen botzko <[email-masked]>
Date: Fri, 16 Jan 2009 14:16:26 -0500

Another use case is connection to a gateway.
H.32x systems do not support out-of-band parameter sets, so interoperability
would be strengthened if in-band repeating is used.
Steve B.

---

From: "Jonathan Lennox" <[email-masked]>
Date: Fri, 16 Jan 2009 14:13:30 -0500

Hi, Ye-Kui,

> in-band a parameter set that has been sent out-of-band, though that is
> not disallowed. Therefore, maybe we don't need recommendations in 8.4
> for in-band repeating of parameter sets out-of-band transmitted?
> 
> The following bullet item in section 8.2.2 prescribes a MUST for in-
> band parameter set transport in such a situation.
> source attribute as specified in section 6.3 of [9], and the answerer
> does not understand the "fmtp" source attribute, in-band transport of
> parameter sets MUST be used, and the answerer MUST NOT include
> "use-level-src-parameter-sets" equal to 1 or the "fmtp" source
> attribute in the answer.
>
> Or do you think we should repeat those parameter set specifications
> from section 8.2.2 in section 8.4?

It should at least be cross-referenced; I didn't notice this text.

It's also not clear to me how an offerer can know whether the answerer
understood the fmtp format or not; there's no explicit message in an
answer corresponding to the fmtp, is there?  This also needs to work
bi-directionally.

The other case that occurred to me is a SIP to H.241 signaling
(non-media-terminating) gateway, where parameter sets can't be
transmitted out-of-band on the far side of the gateway.  I'm wondering
if there needs to be some way that an endpoint can indicate that it
needs in-band parameter sets?

> The text was drafted considering the above MUST for in-band parameter
> set transport. With this in mind, do you still think that something
> more need to be mentioned in section 17?

It couldn't hurt to at least mention it, but it's not required I guess.

> specification. Assuming there are one out-of-band picture parameter
set
> A and one in-band picture parameter set B, both transported during a
> session, and share the same ID. If A and B are assoicated with
different
> access units, then the one associated with the access unit earlier in
> decoding order supersedes the other, at and after the access unit.

Out-of-band parameter sets aren't associated with an access unit, as
such, are they?  I think that if the text said that out-of-band
parameter sets are always associated with an implicit access unit that
comes earlier than any in-band units, that would be sufficient.

> I agree. However, when one source is something like an MCU, behind
> which there are more sources, then the capability parameter sets
> assoicated with the MCU source should be intepreted as the minimum set
> of the union of the MCU source and the sources behind it. I will make
a
> change for this in the next version. Let me know if you have a
different
> opinion.

I'm not quite sure I understand the topology you're describing here.  Is
the middlebox transcoding, or forwarding, sources?

-- 
Jonathan Lennox
Vidyo, Inc
[email-masked]

---

From: Ye-Kui Wang <[email-masked]>
Date: Tue, 20 Jan 2009 03:42:01 -0500

Jonathan, 

> parameter 

OK.

> It's also not clear to me how an offerer can know whether the 
> answerer understood the fmtp format or not; there's no 
> explicit message in an answer corresponding to the fmtp, is 
> there?  This also needs to work bi-directionally.
> 

The presence and value of use-level-src-parameter-sets indicate whether an
entity understands the fmtp fromat (and sprop-level-parameter-sets).

> (non-media-terminating) gateway, where parameter sets can't 
> be transmitted out-of-band on the far side of the gateway.  
> I'm wondering if there needs to be some way that an endpoint 
> can indicate that it needs in-band parameter sets?
> 

Considering this use case, then I think we would need to have a new
parameter, e.g. named in-band-parameter-sets-required. A gateway connected
to some endpoints that are not cabable of doing out-of-band parameter sets
must include this parameter equal to 1 in the answer. I will make this
addition if nobody objects. 

> > > 17 (3984 compatibility)'s discussion of item 7 in the 
> changes from 
> > > 3984.
> > 
> parameter 

If we add the new parameter above, then we need to change the text in
Section 17 anyway.

> > > guidance to a receiver as to what it should do if it, 
> e.g., receives 
> > > out-of-band.  Since mixing in-band updates and 
> out-of-band updates 
> > > in a session is forbidden, it looks like the receiver 
> should assume 
> > > that the out-of-band parameter sets were pre-session, and the 
> > > in-band ones supersede them.
> Out-of-band parameter sets aren't associated with an access 
> unit, as such, are they?  I think that if the text said that 
> out-of-band parameter sets are always associated with an 
> implicit access unit that comes earlier than any in-band 
> units, that would be sufficient.
> 

When parameter sets are transmitted out-of-band during the session, then
synchronization to at least the access unit level (i.e. RTP or NTP timestamp
level) is required. When parameter sets are transmitted out-of-band before
in session setup, then they are put in the beginning of the bitstream by the
receiver and assoicated with the first access unit.

> > > carries multiple independent sources, it's not clear whether the 
> > > max-* parameters (Section 8.1) apply to a receiver's 
> capability for 
> > > a single source, or for its capability for all sources in the 
> > > session.  Based on the descriptions it looks like it's 
> the former, 
> > > since some of the parameters are described in terms of 
> the HRD and 
> > > the like, and in general this makes sense since in such 
> topologies 
> > > it's often impossible for encoders to coordinate so as to stay 
> I'm not quite sure I understand the topology you're 
> describing here.  Is the middlebox transcoding, or 
> forwarding, sources?
> 

I meant the topologies like Topo-Video-switch-MCU [29], which selects and
forwards a source, and does not transcode. 

BR, YK

---

From: Ye-Kui Wang <[email-masked]>
Date: Tue, 20 Jan 2009 03:51:37 -0500

Steve,
 
Right. I think what Roni suggested (i.e. this may be done) should be good
enough. Also, see my previous email (replying Jonathan), the part regarding
addition of a new parameter for such application scenarios.
 
BR, YK

  _____  

From: stephen botzko [mailto:[email-masked]] 
Sent: Friday, January 16, 2009 2:16 PM
To: Ye-Kui Wang
Cc: Roni Even; Jonathan Lennox; IETF AVT WG
Subject: Re: [AVT] WGLC on RFC3984bis: comments

Steve B.

Roni,

BR, YK