Subject: Re: [AVT] SVC payload draft - decoding order recovery for the NI-C and NI-TC modes

From: <[email-masked]>
Date: Wed, 3 Sep 2008 13:03:31 +0300

Thomas, 

Good that you started another thread :-)

>-----Original Message-----
>From: ext Thomas Schierl [mailto:[email-masked]] 
>Sent: 03 September, 2008 12:40
>To: Wang Ye-Kui (Nokia-NRC/Tampere)
>Cc: avt; Even, Roni
>Subject: Re: [AVT] SVC payload draft - decoding order recovery 
>for the NI-C and NI-TC modes
>
>Hi Ye-Kui,
>
>Sure I understand your use cases. What you want to do is interleaving 
>between sessions, i.e. the data is sent in each session in 
>non-interleaved manner, but if putting the data together the data 
>appears as interleaved. But this means, your (inter-session 
>interleaving) receiver must have the same capabilities as a RFC 3984 
>interleaved mode receiver must have. I do not understand, why 
>we want to 
>mix these capabilities with the non-interleaved mode which NI-C 
>(non-interleaved Cross Session DON mode) was intended to be used for.
>

The inention of having the NI-C mdoe (and also the NI-T mode) was to
allow existing RFC 3984 receivers without implementation of the
interleaved-mode to be able to join an SVC multi-session transmission,
not what you said above. 

>Second point, even when you want to do the proposed  interleaving just 
>for the enhanced receiver, which receives the base layer and the 
>enhancement layer, you would still send the base layer data much ahead 
>of time than normally done for the old receiver. Thus you would also 
>need to negotiate the buffer used for inter-session 
>de-interleaving with 
>parameters as "sprop-mst-interleave-depth" "sprop-remux-buf-req" 
>"remux-buf-cap" for the old receiver, since this old receiver 
>now has to 
>cope with the data sent much ahead of time. Sure there is no 
>interleaving for the old receiver but he has to understand the 
>requirement on additional buffering. So you would need an old receiver 
>which is capable of signaling  these parameters. And in that 
>case there 
>is only one backward compatible base layer receiver left: the RFC 3984 
>interleaved mode receiver (packetization mode=2).
>

No, the "old receiver" for receiving the base session simply outputs the
received data to the remultiplexing buffer. It is the remultiplexing
buffer who buffers the data, not the "old receiver", and there is no
additional buffer requirement for the "old receiver".

BR, YK

>Why mixing these things? Why not keeping non-interleaved for receivers 
>with less capability and interleaved for receivers with higher 
>capability?
>
>I propose to remove the parameters "sprop-mst-interleave-depth" 
>"sprop-remux-buf-req" and "remux-buf-cap" from NI-C mode and 
>just allow 
>them to be used with I-C mode.
>
>-- 
>Thomas Schierl
>--------------
>Fraunhofer HHI
>
>
>[email-masked] wrote:
>> Folks,
>>
>> We just had a design team conference call discussing the SVC payload
>> draft
>> 
>(http://www.ietf.org/internet-drafts/draft-ietf-avt-rtp-svc-13.
>txt). One
>> topic discussed was about the decoding order recovery process for the
>> NI-C and NI-TC modes (specified in section 7.1.2 in
>> draft-ietf-avt-rtp-svc-13.txt). In this process, a 
>remultiplexing buffer
>> is used. The remultiplexing buffer takes as input the RTP 
>packets output
>> from the RTP-level reception process for each of the RTP sessions
>> involved in the multi-session transmission. The decoding 
>order recovery
>> process then operates on the remultiplexing buffer. 
>>
>> The topic discussed was why the ecoding order recovery 
>process does not
>> operate directly one NAL units from the RTP-level reception buffer of
>> each RTP session, similarly as in the decoding order recovery process
>> for the NI-T and NI-TC modes (specified in section 7.1.1 in
>> draft-ietf-avt-rtp-svc-13.txt), and what is the usefulness of the
>> related parameters like sprop-mst-interleave-depth and
>> sprop-remux-buf-req. I have explained the use cases during the
>> conference call, and was requested to send an email to this 
>mailing with
>> the explanation, in order to have an open discussion.
>>
>> So here are the use cases and the use of the parameters 
>related to the
>> remultiplexing buffer. 
>>
>> Use case 1: robust scheduling. The sender transmits more 
>important data
>> in the lower layers conveyed in lower RTP sessions earlier than less
>> important data in the higher layers conveyed in higher RTP sessions,
>> thus allowing a better chance to receive more important data, e.g. by
>> allowing retransmission of the more important data. As a result, the
>> receiver needs to use extra buffer for higher layer data 
>before starting
>> the decoding order recovery process. The parameters define 
>the minimum
>> size of the buffer required. 
>>
>> Use case 2: constant bitrate transmission. In many application
>> scenarios, e.g. DVB-H, the pipes for conveying data from 
>different RTP
>> sessions are of constant bandwidth. However, the instant 
>video bitrates
>> for different layers are by nature variable over time. This 
>is more true
>> in SVC. For example, for a video bitstream containing regular random
>> access points, which are required in multicast and broadcast
>> applications to enable new comers to tune in and program channel
>> switching, IDR pictures are typically encoded at the random access
>> points. For the base layer, which is H.264/AVC compatible, 
>IDR pictures
>> are true intra pictures. For the enhancement layers, IDR pictures can
>> still be encoded using inter-layer prediction hence the coded size is
>> way smaller than the base layer intra picture at the same access unit
>> (it is common that an intra coded picture is 10 time larger 
>as an inter
>> coded picture). Therefore, the bitrate of an enhancement layer can be
>> much smoother than the bitrate of the base layer. As a result, to let
>> the data pass the pipes, the transmitter has to transmit data of the
>> base layer after the IDR picture much later than the data of an
>> enhancement layer at the same access unit. Similarly as above, the
>> receiver needs to use extra buffer for enhancement layer data before
>> starting the decoding order recovery process. The parameters 
>define the
>> minimum size of the buffer required. 
>>
>> Use case 3: use of NI-MTAP packets. The newly specified 
>NI-MTAP packet
>> allows aggregation of data from multiple access units in a
>> non-interleaved manner within a session. However, when the use of
>> NI-MTAPs is not aligned among all the RTP sessions (e.g. 
>this is always
>> the case when the base RTP session is RFC 3984 compatible - 
>which is a
>> requirement per the payload draft), the receiver needs to use extra
>> buffer for the RTP session that does not use NI-MTAP or that uses
>> NI-MTAP but aggregating data from less access units before 
>starting the
>> decoding order recovery process. The parameters define the 
>minimum size
>> of the buffer required.
>>  
>> BR,YK
>> _______________________________________________
>> Audio/Video Transport Working Group
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/avt
>>
>>
>>   
>
>
>
>----
>Visit us at
>
>IBC 2008 / Amsterdam (NL), 12 - 16 September 2008 / Hall 8 Booth C81
>http://www.hhi.fraunhofer.de/index.php?id=2399&L=1
>
>insite 2008 / Johannesburg (South Africa), Sandton Convention 
>Centre / 15 - 17 September 2008 
>http://www.insitex.co.za
>
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt

---

From: Thomas Schierl <[email-masked]>
Date: Wed, 03 Sep 2008 11:39:54 +0200

Hi Ye-Kui,

interleaved mode receiver must have. I do not understand, why we want to 

Second point, even when you want to do the proposed  interleaving just 
for the enhanced receiver, which receives the base layer and the 
enhancement layer, you would still send the base layer data much ahead 
of time than normally done for the old receiver. Thus you would also 
need to negotiate the buffer used for inter-session de-interleaving with 
parameters as "sprop-mst-interleave-depth" "sprop-remux-buf-req" 
"remux-buf-cap" for the old receiver, since this old receiver now has to 
cope with the data sent much ahead of time. Sure there is no 
interleaving for the old receiver but he has to understand the 
requirement on additional buffering. So you would need an old receiver 
which is capable of signaling  these parameters. And in that case there 
is only one backward compatible base layer receiver left: the RFC 3984 
interleaved mode receiver (packetization mode=2).

Why mixing these things? Why not keeping non-interleaved for receivers 
with less capability and interleaved for receivers with higher capability?

I propose to remove the parameters "sprop-mst-interleave-depth" 
"sprop-remux-buf-req" and "remux-buf-cap" from NI-C mode and just allow 
them to be used with I-C mode.

-- 
Thomas Schierl
--------------
Fraunhofer HHI

[email-masked] wrote:
> (http://www.ietf.org/internet-drafts/draft-ietf-avt-rtp-svc-13.txt). One
> draft-ietf-avt-rtp-svc-13.txt). In this process, a remultiplexing buffer
> is used. The remultiplexing buffer takes as input the RTP packets output
> involved in the multi-session transmission. The decoding order recovery
> The topic discussed was why the ecoding order recovery process does not
> conference call, and was requested to send an email to this mailing with
> So here are the use cases and the use of the parameters related to the
> remultiplexing buffer. 
>
> Use case 1: robust scheduling. The sender transmits more important data
> receiver needs to use extra buffer for higher layer data before starting
> the decoding order recovery process. The parameters define the minimum
> scenarios, e.g. DVB-H, the pipes for conveying data from different RTP
> sessions are of constant bandwidth. However, the instant video bitrates
> for different layers are by nature variable over time. This is more true
> points. For the base layer, which is H.264/AVC compatible, IDR pictures
> (it is common that an intra coded picture is 10 time larger as an inter
> starting the decoding order recovery process. The parameters define the
> Use case 3: use of NI-MTAP packets. The newly specified NI-MTAP packet
> NI-MTAPs is not aligned among all the RTP sessions (e.g. this is always
> the case when the base RTP session is RFC 3984 compatible - which is a
> NI-MTAP but aggregating data from less access units before starting the
> decoding order recovery process. The parameters define the minimum size
> of the buffer required.
>  
> BR,YK

----
Visit us at

insite 2008 / Johannesburg (South Africa), Sandton Convention Centre / 15 - 17 September 2008 
http://www.insitex.co.za
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt