Subject: RE: [AVT] FW: I-D Action:draft-ietf-avt-rtp-svc-04.txt

From: "Thomas Schierl" <[email-masked]>
Date: Wed, 2 Jan 2008 18:35:39 +0100

Hi Colin,

Ok, I will try to fix the formatting before the conference call next week.

Happy new year!
Thomas

> -----Original Message-----
> From: Colin Perkins [mailto:[email-masked]]
> Sent: Sunday, December 30, 2007 7:59 PM
> To: Thomas Schierl
> Cc: [email-masked]; 'Even, Roni'; 'Stephan Wenger'; [email-masked]
> Subject: Re: [AVT] FW: I-D Action:draft-ietf-avt-rtp-svc-04.txt
>
> Thomas,
>
> On 18 Dec 2007, at 18:15, Thomas Schierl wrote:
> > I just submitted the -04 version of the RTP payload draft for SVC.
> > Besides other changes, I corrected the formatting of the media type
> > section.
>
> There are so many formatting changes and problems with this version
> it's very difficult to find the technical changes. Please can you
> ensure that future versions of the draft have consistent formatting,
> such that diff tools can be used (the xml2rfc tool is extremely
> useful here).
>
> Thanks,
> --
> Colin Perkins
> http://csperkins.org/

----
Visit us at

FOE 2008 (8th Fiber Optics Expo) / Tokio Big Sight, Japan / 16 - 18 January 2008 / Booth 2 - 50
http://www.foe.jp/english 

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: "Even, Roni" <[email-masked]>
Date: Thu, 3 Jan 2008 15:15:07 +0200

Thomas,

I am reading the svc draft and as long as you address non technical
parts I noticed that the definitions in section 5.1 are different from
the SVC (H.264 amendment 3) for the same terms. I suggest that the
definitions should be the same or use different names.

The same comment for section 3.3 comparing to G.7.4.1.1 for the U and RR
bits specifically.

Roni

> From: Thomas Schierl [mailto:[email-masked]]
> Sent: Wednesday, January 02, 2008 7:36 PM
> To: 'Colin Perkins'
> Cc: [email-masked]; Even, Roni; 'Stephan Wenger'; [email-masked]
> Subject: RE: [AVT] FW: I-D Action:draft-ietf-avt-rtp-svc-04.txt
> 
> Hi Colin,
>

---

From: <[email-masked]>
Date: Fri, 4 Jan 2008 01:46:01 +0200

Hi Roni,

Thanks for reading the new draft and for your new comments. 

- Our intention was to summarize the definitions in a way easier to
understand - that's why we did not copy those definitions from the SVC
coding spec. However, if you (and others don't disagree) would like
those definitions to be phrased same as in the SVC coding spec, we can
certainly do that. I will do this change to the next version if no one
disagrees. 

- Similarly as above, the semantics of the U bit was phrased as
currently in the payload draft also to make it easier to understand. The
semantics from the SVC spec is as follows:

use_ref_base_pic_flag equal to 1 specifies that reference base pictures
are used as reference pictures for the inter prediction process.
use_ref_base_pic_flag equal to 0 specifies that decoded pictures are
used as reference pictures during the inter prediction process.
The values of use_ref_base_pic_flag shall be the same for all NAL units
of a dependency representation. use_ref_base_pic_flag shall be equal to
0 in filler prefix NAL units.

While the current semantics in the payload draft is as follows:

use_ref_base_pic_flag.  A value of 1 indicates that only reference base
pictures are used during the inter prediction process.  A value of 0
indicates that the reference base pictures are not used during the inter
prediction process. 

Except for the last sentence of the semantics in the SVC spec (which is
a limit to a conforming bitstream therefore no need to be mentioned in
the payload draft), both semantics are equivalent. In the payload draft,
I avoided having the sentence "use_ref_base_pic_flag equal to 0
specifies that decoded pictures are used as reference pictures during
the inter prediction process.", which could be confusing to people not
deeply familiar with the SVC spec, wherein "decoded picture" in SVC
context refers to the reconstructed picture used for output, while
another (optional) reconstructed picture from decoding of the same
access unit, "reference base picture", used for inter prediction but not
for output, is not a "decoded picture". I have checked the SVC spec,
found that a new definition of "decoded picture" is unfortunately not
present in Annex G.3. 

I therefore think it is a little bit better to have the current
semantics in the payload draft, unless a new definition of "decoded
picture" is added in the SVC spec (Annex G).  

- Regarding the RR bits, if you meant for the difference between
"Receivers SHOULD discard NAL units with RR not equal to '11'." in the
payload draft and "Decoders shall ignore the value of
reserved_three_2bits." in the SVC spec, then I would say that having the
difference is correct, because herein the intention is to specify the
RTP receiver's operation not the decoder's operation, and "SHOULD"
should be used in stead of "SHALL".

BR, YK 

>From: ext Even, Roni [mailto:[email-masked]] 
>Sent: Thursday, January 03, 2008 3:15 PM
>To: Thomas Schierl; Colin Perkins
>Cc: [email-masked]; [email-masked]
>I am reading the svc draft and as long as you address non 
>technical parts I noticed that the definitions in section 5.1 
>are different from the SVC (H.264 amendment 3) for the same 
>terms. I suggest that the definitions should be the same or 
>use different names.
>
>
>The same comment for section 3.3 comparing to G.7.4.1.1 for 
>the U and RR bits specifically.
>
>
>Roni
>
>> FOE 2008 (8th Fiber Optics Expo) / Tokio Big Sight, Japan / 16 - 18 
>> January 2008 / Booth 2 - 50 http://www.foe.jp/english
>

---

From: "Even, Roni" <[email-masked]>
Date: Fri, 4 Jan 2008 15:21:51 +0200

YK,

There is a major difference here. According to the text in SVC you do
not discard NAL which do not have 11 you just ignore the field.

Roni

> From: [email-masked] [mailto:[email-masked]]
> Sent: Friday, January 04, 2008 1:46 AM
> To: Even, Roni; [email-masked]; [email-masked]
> >> Ok, I will try to fix the formatting before the conference call
next
> >week.

---

From: <[email-masked]>
Date: Fri, 4 Jan 2008 16:36:55 -0000

I have just finished reading this draft for the first time, and have the
following comments on it. I have not been involved in the development of
this draft, and so only have the written words to use to try to
understand it.

1) R and RR bits, pages 8 and 9
As has been noted by others, these are defined differently here to SVC.
Here the NAL unit should be discarded, whereas in SVC the bits are just
ignored.

Encouraging discarding of such NALs does not seem to be a good idea, as
it makes extensions to SVC harder as any use of these bits in SVC would
cause old receivers to discard NAL units that they might be able to use.

2) Base layer and full base layer, page 11
As others have stated, I also think that base layer should be defined as
in SVC: doing anything else is simply confusing to the reader. In
addition there are some possible incorrect usages of base layer, such as
in section 4 on page 9. "T0 base layer" is a good term to use in place
of the current "base layer".

3) CL-DON definition, page 11
I find it confusing that the term CL-DON is used both for an optional
syntax element and for a derived variable. I think the text could be
made much clearer if different names were used. Also there are only
short descriptions, scattered throughout the draft, of how and when this
derivation process is performed and what is done with the resulting
numbers. 

4) Semantic definitions, page 16
A number of the definitions on this page refer to "the remaining NAL
units in the payload". But NAL units form the payload of the PACSI and
form the payload of the aggregation packet. It would help if it were
clarified which of these payloads were intended - presumably the latter.

5) A bit, page 17
I find it difficult to understand this definition. It might help to
define target NAL units outside of this particular definition first as
it is used elsewhere. I have problems with the consistency of the
definition and the note below it. If some of the NAL units belonging to
the anchor layer representations were in an earlier aggregation packet,
but were not target NAL units in that packet, the A bit being set in the
current aggregation packet would not actually indicate a switching
point?

6) T bit, page 18
Is this the same T bit that indicates that the CL-DON field is present?
I would imagine this was not intended, but if it were, then
clarification should be added, and all of the definitions of the T bit
brought together into one place.

7) P bit, page 18
Can redundant and non-redundant data be included in the same aggregation
packet? Including a redundant representation of a previous picture with
the primary encoding of the current picture in one aggregation packet is
a useful thing to do (and one we (BT) have done with audio in the past).
Would a better definition of the P bit be that all NAL units in the
aggregation packet contain redundant data? Or is a note sufficient, to
remind the implementer to ensure that redundant NAL units are not the
target NAL units in an aggregation packet which also contains
non-redundant data?

8) SEI NAL units in PACSI, page 19
I have doubts about the paragraph "SEI NAL units included in the PACSI
NAL unit, if any, MUST contain a subset of the SEI messages associated
with the access unit of the first NAL unit following the PACSI NAL unit
within the aggregation packet.". As a subset could presumably be an
empty set, does this paragraph actually say anything at all? Which SEI
messages must be included? And are any excluded, such as those
associated with some other access unit?

9) Options I and II, page 21
It is stated that there are two options, I and II, but there is no
identification of these options in the text. Presumably I starts
immediately and II starts with "CL-DON must ..."

10) Informative note, page 21
I do not find that this informative note lives up to its name, finding
it not at all informative! It refers to a restriction "a" that does not
exist, and even after reading the rest of it many times, I am still
confused.

11) Option II, page 21
"STAP-A must be used". So no fragmentation is allowed?

12) "rage", page 23
Should be "range".

13) Payload format parameters, page 41
It is not clear to me why some of these parameters should not be used
when CL-DON (syntax element?) is present. Are these parameters only
useful if option I is used?

14) sprop-prebuf-size, page 41
My reading of this definition is that it is stating that the buffer size
for inter session re-ordering depends only on the NAL units of the
current session. Is this right? And if so, why is this the case?

15) sprop-init-buf-time  and sprop-prebuf-time, page 41 (and page 38)
It is not clear to me how these are related to each other. Should they
be combined? Or does one override the other?

16) Example parameters, page 50
On the third line, fmtp:97 should be fmtp:96 ?

17) General issues
a) Presumably the document applies to both the case of using and not
using session multiplexing. Also, having read it twice, I am coming to
the conclusion that when session multiplexing is used, the same
packetisation mode (non-interleaved, interleaved) must be used in all
sessions. It would help the reader a lot if these top level scenarios
were more clearly described, and then later how the various operations,
such as restoring decoding order across multiple sessions can be carried
out in the various scenarios.

b) Regarding the non-interleaved mode, I have difficulty understanding
the implications of the two options for restoring decoding order -
hopefully this will become clearer in the near future.

For information, and risking confusing the situation even further, we in
BT implemented McCanne's receiver driven layer multicast concept using
H.263 Annex O in 1998-1999. The implementation was published at the IEE
Conference on Distributed Imaging, 18-19 November 1999 and at the Packet
Video Workshop 2000, Cagliari, Italy. The former paper and its
PowerPoint presentation are at:
http://ieeexplore.ieee.org/iel5/6724/17984/00831192.pdf?arnumber=831192,
where section 3.3 describes how we overcame the problem of restoring
decoding order over multiple sessions: we used a sequence number that
was sequential across the RTP sessions in addition to the RTP sequence
number that is sequential within a session, and carried that extra
(cross layer) sequence number using the RTP header extension mechanism.

Best regards

Mike

Mike Nilsson 
Multimedia Analysis and Coding
BT Group Chief Technology Office
___________________________ 

Sirius House (B54-MH), Room 92 
Adastral Park, Martlesham Heath, Ipswich, IP5 3RE, UK 
Tel:    +44 1473 645413 
Mobile: +44 7917 025433 
Fax:    +44 1908 862365 
Email:  [email-masked]

-----Original Message-----
From: Even, Roni [mailto:[email-masked]] 
Sent: 04 January 2008 13:22
To: [email-masked]; [email-masked]; [email-masked]
Cc: [email-masked]; [email-masked]
Subject: RE: [AVT] FW: I-D Action:draft-ietf-avt-rtp-svc-04.txt 

YK,

Roni

next

---

From: <[email-masked]>
Date: Fri, 4 Jan 2008 20:43:42 +0200

Hi Roni,

Ah, I overlooked this. You are right, we should say "Receivers SHOULD
ignore the value of RR." 

BR, YK 

>Sent: Friday, January 04, 2008 3:22 PM
>To: Wang Ye-Kui (Nokia-OCTO/Tampere); [email-masked]; 
>[email-masked]
>- Regarding the RR bits, if you meant for the difference 
>between "Receivers SHOULD discard NAL units with RR not equal 
>to '11'." in the payload draft and "Decoders shall ignore the 
>value of reserved_three_2bits." in the SVC spec, then I would 
>say that having the difference is correct, because herein the 
>intention is to specify the RTP receiver's operation not the 
>decoder's operation, and "SHOULD"
>There is a major difference here. According to the text in SVC 
>you do not discard NAL which do not have 11 you just ignore the field.
>
>Roni
>
>> >parts I noticed that the definitions in section 5.1 are different 
>> >from the SVC (H.264 amendment 3) for the same terms. I suggest that 
>> >the definitions should be the same or use different names.
>> >
>> >

---

From: <[email-masked]>
Date: Fri, 4 Jan 2008 23:27:32 +0200

Hi Mike,

Thanks for your careful review and valuable comments. See my responses
inline. I have left some comments for Thomas (Schierl). 

BR, YK 

>From: ext [email-masked] [mailto:[email-masked]] 
>Sent: Friday, January 04, 2008 6:37 PM
>To: [email-masked]
>I have just finished reading this draft for the first time, 
>and have the following comments on it. I have not been 
>involved in the development of this draft, and so only have 
>the written words to use to try to understand it.
>
>Here the NAL unit should be discarded, whereas in SVC the bits 
>are just ignored.
>
>Encouraging discarding of such NALs does not seem to be a good 
>idea, as it makes extensions to SVC harder as any use of these 
>bits in SVC would cause old receivers to discard NAL units 
>that they might be able to use.
>

Agreed. We should say ignore the value as in my earlier email replying
Roni. 

>2) Base layer and full base layer, page 11 As others have 
>stated, I also think that base layer should be defined as in 
>SVC: doing anything else is simply confusing to the reader. In 
>addition there are some possible incorrect usages of base 
>layer, such as in section 4 on page 9. "T0 base layer" is a 
>good term to use in place of the current "base layer".
>

Agreed. But in section 4 we intended not to use "full base layer", to
allow layered multicasting of an AVC stream that is temporally scalable.

>I find it confusing that the term CL-DON is used both for an 
>optional syntax element and for a derived variable. I think 
>the text could be made much clearer if different names were 
>used. Also there are only short descriptions, scattered 
>throughout the draft, of how and when this derivation process 
>is performed and what is done with the resulting numbers. 
>

OK - will improve to avoid the flaws you pointed out. 

>A number of the definitions on this page refer to "the 
>remaining NAL units in the payload". But NAL units form the 
>payload of the PACSI and form the payload of the aggregation 
>packet. It would help if it were clarified which of these 
>payloads were intended - presumably the latter.
>

Yes your understanding is correct - will make it clearer. 

>5) A bit, page 17
>I find it difficult to understand this definition. It might 
>help to define target NAL units outside of this particular 
>definition first as it is used elsewhere. I have problems with 

OK - this make the definition of "target NAL units" outside this
semantics. 

>the consistency of the definition and the note below it. If 
>some of the NAL units belonging to the anchor layer 
>representations were in an earlier aggregation packet, but 
>were not target NAL units in that packet, the A bit being set 
>in the current aggregation packet would not actually indicate 
>a switching point?
>

Earlier (in -02) we specified the semantics to cover also NAL units
belonging to the same anchor layer representation but not contained in
the current aggregation packet. Then there was a comment from Magus
Westerlund saying that the semantics should not be more than local to
the current aggregation packet. We thus made the semantics change as is
now and added a SHOULD requirement (the last sentence of the semantics)
to impose the requirement to NAL units not contained in the current
aggregation packet. 

The improvement I can think of is to explicitly state the situation in
the note. I'd appreciate if you (or anyone else) have a better
suggestion. 

>clarification should be added, and all of the definitions of 
>the T bit brought together into one place.
>

Oops! We intended to remove the second semantics (about temporal layer
switching point) but failed. Initially we used the T bit for indicating
of temporal layer switching point. But later we found that one bit is
needed to indicate the presence of CL-DON, and all other bits seem to be
more important. 

>7) P bit, page 18
>Can redundant and non-redundant data be included in the same 
>aggregation packet? Including a redundant representation of a 

It is not disallowed. 

>previous picture with the primary encoding of the current 
>picture in one aggregation packet is a useful thing to do (and 
>one we (BT) have done with audio in the past).

Hmm, makes sense. 

>Would a better definition of the P bit be that all NAL units 
>in the aggregation packet contain redundant data? Or is a note 
>sufficient, to remind the implementer to ensure that redundant 
>NAL units are not the target NAL units in an aggregation 
>packet which also contains non-redundant data?
>

Good point again - I like the former (definition of the P bit be that
all NAL units in the aggregation packet contain redundant data). Will
integrate this change if no one disagrees. 

>I have doubts about the paragraph "SEI NAL units included in 
>the PACSI NAL unit, if any, MUST contain a subset of the SEI 
>messages associated with the access unit of the first NAL unit 
>following the PACSI NAL unit within the aggregation packet.". 
>As a subset could presumably be an empty set, does this 
>paragraph actually say anything at all? Which SEI messages 
>must be included? And are any excluded, such as those 

Here we are trying to say the following:
- Never include SEI messages associated with other access units than the
one (target access unit) containing the first NAL unit following the
PACSI in the aggregation packet. 
- A subset (zero to all) of the SEI messages associated with the target
access unit can be included in the PACSI. 

Please let me know it the above clarification does not resolve your
comment. 

>It is stated that there are two options, I and II, but there 
>is no identification of these options in the text. Presumably 
>I starts immediately and II starts with "CL-DON must ..."
>
>I do not find that this informative note lives up to its name, 
>finding it not at all informative! It refers to a restriction 
>"a" that does not exist, and even after reading the rest of it 
>many times, I am still confused.
>

I leave the above 2 comments to Thomas. 

>"STAP-A must be used". So no fragmentation is allowed?

In option II, for each session in the layered multicast, either
packetization mode 2 or 1 can be used. If packetization mode 2 is used,
then anything as allowed for mode 2 in RFC 3984 (including fragment
units) can be used, with only one restriction - the DON field must
indicate CL-DON. If packetization mode 1 is used, then only STAP-A can
be used, meaning fragment units cannot be used. 

>It is not clear to me why some of these parameters should not 
>be used when CL-DON (syntax element?) is present. Are these 
>parameters only useful if option I is used?
>
>My reading of this definition is that it is stating that the 
>buffer size for inter session re-ordering depends only on the 
>NAL units of the current session. Is this right? And if so, 
>why is this the case?
>
>15) sprop-init-buf-time  and sprop-prebuf-time, page 41 (and 
>page 38) It is not clear to me how these are related to each 
>other. Should they be combined? Or does one override the other?
>

I leave the above 5 comments to Thomas. 

>
>17) General issues
>a) Presumably the document applies to both the case of using 
>and not using session multiplexing. Also, having read it 
>twice, I am coming to the conclusion that when session 
>multiplexing is used, the same packetisation mode 
>(non-interleaved, interleaved) must be used in all sessions. 
>It would help the reader a lot if these top level scenarios 
>were more clearly described, and then later how the various 
>operations, such as restoring decoding order across multiple 
>sessions can be carried out in the various scenarios.
>

Agree. I hope my above explanation regarding option II is helpful for
your understanding. Will address this after the open issue regarding
options I and II is resolved. 

>b) Regarding the non-interleaved mode, I have difficulty 
>understanding the implications of the two options for 
>restoring decoding order - hopefully this will become clearer 
>in the near future.
>

I believe the thing will become clearer after the (biggest) open issue
is resolved. 

>For information, and risking confusing the situation even 
>further, we in BT implemented McCanne's receiver driven layer 
>multicast concept using
>H.263 Annex O in 1998-1999. The implementation was published 
>at the IEE Conference on Distributed Imaging, 18-19 November 
>1999 and at the Packet Video Workshop 2000, Cagliari, Italy. 
>The former paper and its PowerPoint presentation are at:
>where section 3.3 describes how we overcame the problem of 
>restoring decoding order over multiple sessions: we used a 
>sequence number that was sequential across the RTP sessions in 
>addition to the RTP sequence number that is sequential within 
>a session, and carried that extra (cross layer) sequence 
>number using the RTP header extension mechanism.
>

This is very useful information - thanks!

[end]

---

From: "Thomas Schierl" <[email-masked]>
Date: Tue, 8 Jan 2008 15:54:49 +0100

Hi Mike, Ye-Kui, all,

See my comments inline. But most of the issues seems to be related to the
formatting problems of the -04 version (current version: -05). Sorry again
for that.

Kind regards,
Thomas

> Sent: Friday, January 04, 2008 10:28 PM
> To: [email-masked]; [email-masked]

This is due to the formatting problem. It should be clear in version -05.

> >many times, I am still confused.

"a" is visible in -05.

This is an additional informative note on the constraints shown above, under
point I.). See -05 version.

> be used, meaning fragment units cannot be used.

Yes.

> >parameters only useful if option I is used?

Currently, there is no explicit signaling whether non-DON mode (I.) or
CL-DON mode (II.) is used for rearranging decoding order in session
multiplexing scenarios. This is a first attempt to differentiate between the
two modes. But, if the two modes should stay, we need signaling for that.

> >NAL units of the current session. Is this right? And if so,

The text needs work. I will try to calrify in -06.

> >other. Should they be combined? Or does one override the other?

TBD. A combination should still lead to correct rearranging of decoding
order.

> >On the third line, fmtp:97 should be fmtp:96 ?

Yes. Included in -05

> >> >> > > Besides other changes, I corrected the formatting of the
> media
> >> >type
> >> >> > > section.
> >> >> >

----
Visit us at

FOE 2008 (8th Fiber Optics Expo) / Tokio Big Sight, Japan / 16 - 18 January 2008 / Booth 2 - 50
http://www.foe.jp/english 
GSMA Mobile World Congress / Barcelona, Spain / 11 - 14 February 2008 / Broadcast Mobile Convergence Forum / Booth 2B82
http://www.mobileworldcongress.com/homepage.htm