Subject: RE: [AVT] draft-ietf-avt-rtp-h264-01.txt in band PS

From: [email-masked]
Date: Thu, 13 Mar 2003 15:30:41 +0100

Roni,

Thank you for bringing this up.

Stephan,

After reflexion the sentence in the draft:

"    Although, conceptually, the Parameter Set updates are not designed
    to be sent in the synchronous packet stream, this memo contains
    means to convey them in the RTP packet stream.   "

looks very dangerous to me.

let me explain.

*************************************************************************

H.264 has been designed with the capability to  change the configuration
dynamically and possibly very often (at every frame!) I wish it was clear for
everyone that this directly impacts the concept of  "Random Access Point",
which is very different depending on applications:

I assume you want to use the "fast update request" to send the parameter set ID(s)
of the missing PS when a client "jumps" into a stream so as to cause the sender to insert
the relevant (missing) PS in the stream.

I conclude that in your application it is acceptable to have a "black screen" while
the decoder waits for the full list of useful PS(s)....

BTW I am concerned by the possibility  that -since each single frame of the stream
may have a different PS  and that H264 supports 8k different PS for a stream-
you may spend the first minutes sending requests for missing PS(s)
and in the meanwhile display either black or a bad slide show!

but Ok, this is your application right ?

Now thinking about that made me push the reasoning one more step:
It looks to me that H264 has pushed the flexibility so far that using
"fast update request" to get PS(s) can rapidly become
more like resorting to retransmission in case of "vital" packet losses
... a bad idea usually :-(

I.e. the problem of random access generalizes to an error resilience issue,
and I have to insist that:

1) in band PS is *bad* for error resilience

2) and that this is true for *all applications* using non-reliable transport e.g. RTP

***************

The obvious *safe* alternative is to  include all the required PS(s) in the initial session setup.

***************

Anyway ...let me translate this to the VOD media distribution case.

A) a first concern is the conformance of a decoder with regard to the support of PS in band.

We need to scope it in the RTP payload  because
implementing a VOD system that supports PS inband is
*much* more difficult  and *not desirable* as follows:

B) with video on demand you would get in trouble when a media player
performs a jump and discovers it cannot decode the video
it is receiving because it misses some PS(s) that are embedded in the stream
"before" the jump !

there are 3 solutions there

* jump on "complete" PS(s) updates only (unefficient and unpractical)

* implement a dedicated trick in the streaming server so that it scans
the stream and looks up all the required PS(s) (ouch!) to include them in
some kind of "primer" in the first RTP packets (and risk to loose them!)

* do not use in band PS, put all the required PS(s) in the SDP instead!

So you understand the choice is simple: for VOD we dont want in band PS !

***********

Conclusion: I can only marginaly understand you may want to have in band PS
but there MUST be a way to make conformant systems that do not need to implement it,

therefore that sentence should be:

"This memo contains means to convey Parameter Set updates in the RTP packet stream;
but since, conceptually, these are not designed to be sent in the synchronous packet stream,
supporting in band Parameter Set transmission is OPTIONAL".

regards,

Philippe Gentric
Software Architect
Philips MP4Net
[email-masked]
http://www.platform4.philips.com

                                                                                                                                       
                                                   To:   "'Stephan Wenger'" <[email-masked]>                                    
                                                    [email-masked]                                                                       
                                                   cc:   (bcc: Philippe Gentric/MP4-SUR/CE/PHILIPS)                                    
                                                   Subject:    RE: [AVT] Updated issues to discuss -- draft-ietf-avt-rtp-h264-01       
               "Even, Roni"                         .txt                                                                               
               <[email-masked]                                                                                                
               >                                   Classification:                                                                     
                                                                                                                                       
               Sent by:                                                                                                                
               [email-masked]                                                                                                      
                                                                                                                                       
               13/03/2003 11:22                                                                                                        

Hi,
I would like to point out that there are problems with supporting parameters
sets in multipoint video conferencing scenarios. When using out of band
signalling for the parameter sets if a video switch need to occur then there
is a need for a new parameter set for the new video stream and it has to be
available to the receiver before the actual stream.
The current work at the ITU suggests to use in band parameter sets and to
send the parameter sets before the full picture as a response to a fast
update request. (available in the RTCP feedback draft)
Regards
Roni Even

> -----Original Message-----
> From: Stephan Wenger [mailto:[email-masked]]
> Sent: Thursday, March 13, 2003 11:38 AM
> To: [email-masked]
> Subject: [AVT] Updated issues to discuss --
> draft-ietf-avt-rtp-h264-01.txt
>
>
> Folks,
> here is an updated list of the issues re the H.264/JVT
> packetization.  It
> reflects mostly the discussions here in Pattaya, where MPEG and JVT
> currently meets:
>
> Unchanged items:
> 1. Status of JVT -- should be out for the final ballot by the
> time of the
> meeting.
> 2. Fragmentation -- we decided on adding application layer
> fragmentation of
> NALUs to support some exotic scenarios.  I plan to describe
> the pros and
> cons of the fragmentation, and hope for more guidance from
> the meeting re
> these feature.  We had discussed this briefly in Atlanta
> already, but it
> has turned out that I forgot some applications where
> fragmentation is useful.
> 3. MIME/SDP sections: are they ok?
> 4. Interaction with MPEG-4 simple packetization (since H.264 is also
> ISO/IEC 14496-10).  Miska and I are currently in Pattaya at
> the JVT/MPEG
> meeting, as are Jan Van-der-Meer, Dave Singer, Young-Kown Lim
> (and other
> knowledgeable people).  We will have some discussions about
> the interaction
> of the two drafts, and report probably to the mailing list,
> but certainly
> orally at the meeting, about the outcome of these discussions.
> 5. I currently believe that the problem mentioned in section
> 12 of the
> draft doesn't exist, but we will discuss this here in Pattaya
> as well and
> take action as necessary.
> 6. Security section seems to need more work, but this is not
> reflected in
> the draft yet.  We plan to present a summary of the problems
> and proposed
> solution if there is time left.
>
> New items:
> 7. Interoperability between MPEG-4 simple and H.264
> packetization -- as it
> has turned out, we already have one common operation point.
> Now that AUs
> are syntactically defined in MPEG-4, we can add something
> similar to a STAP
> that adds another common operation point.  Our goal is to
> make this draft a
> binary-compatible superset of the MP4-Simple draft when
> carrying H.264, but
> it is unclear whether this can be achieved.
> 8. MIME/SDP codepoints to support out-of-level picture sizes
> -- the video
> conferencing industry needs to support extreme picture sizes
> (e.g. 1500 x
> 1000 pels) at low frame rates, for example for document
> cameras.  However,
> they don't want to signal a profile/level combination that
> allows for this
> picture size (memory requirements), because that also implies
> the ability
> to decode high speed bit streams at high frame rates (computational
> requirements).  Such codepoints are in principle agreed in
> the ITU-T H.245
> protocol, and we are asked to add similar codepoints in the MIME/SDP.
> 9. I will be prepared to explain DON (why it is there, and it
> works), but
> we admit that there is an urgent need for clearer language for both.
>
> Regards,
> Stephan
>
> _______________________________________________
> Audio/Video Transport Working Group
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/avt
>
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: "Even, Roni" <[email-masked]>
Date: Thu, 13 Mar 2003 17:15:47 +0200

Hi Phillippe,

About your VOD example. I do not see the difference between the PS and the
full picture. If you skip frames you may need to look for some full frame if
you the new frame is not referencing a full frame that is in the decoder
buffers. In the case of Parameter Sets it is even simpler since the right
way will be to send all the parameter sets you intend to use in the
beginning of the connection between the media player client and the server.

As for error resilience I am OK with your comment.

The number of parameter sets is not the only issue but also if you have for
example a multicast group and one end point is transmitting it may have used
3 parameter sets, than a different end point is transmitting using the same
parameter sets but their order in the set is different. There is a need to
send the new parameter sets.

As for the paragraph I would like to have it as

"This memo contains means to convey Parameter Set updates in the RTP packet
stream;
supporting of in band Parameter Set transmission is application dependent.
When using in band Parameter Set, the Parameter sets should be sent as part
of the response  to an RTCP picture loss indication ".

Regards
Roni Even

> From: [email-masked] 
> [mailto:[email-masked]]
> Sent: Thursday, March 13, 2003 4:31 PM
> To: Even, Roni
> Cc: [email-masked]; 'Stephan Wenger'
> Subject: RE: [AVT] draft-ietf-avt-rtp-h264-01.txt in band PS
> 
> 
> Roni,
> 
> The obvious *safe* alternative is to  include all the 
> required PS(s) in the initial session setup.
> 
> ***************
> 
>                                                    To:   
> "'Stephan Wenger'" <[email-masked]>                    
>                 
>                                                     
> [email-masked]                                                  
>                      
>                                                    cc:   
> (bcc: Philippe Gentric/MP4-SUR/CE/PHILIPS)                    
>                 
>                                                    Subject:   
>  RE: [AVT] Updated issues to discuss -- 
> draft-ietf-avt-rtp-h264-01       
>                "Even, Roni"                         .txt      
>                                                               
>            
>                <[email-masked]                       
>                                                               
>            
>                >                                   
> Classification:                                               
>                       
>                                                               
>                                                               
>            
>                Sent by:                                       
>                                                               
>            
>                [email-masked]                             
>                                                               
>            
>                                                               
>                                                               
>            
>                13/03/2003 11:22                               
>                                                               
>            
>                                                               
>                                                               
>            
>                                                               
>                                                               
>            
> 
> 
> 
> 
> Hi,

---

From: [email-masked]
Date: Thu, 13 Mar 2003 16:24:55 +0100

Roni, you write:

> In the case of Parameter Sets it is even simpler since the right

Ah but that is *exactly* the point !

I know I need to convey all the required parameter sets to my decoder *before* starting to decode

My point is that since I have 2 ways to do it:
* out of band and reliable (typically SDP  on  TCP)
or
* in band and not reliable

I prefer to do it out of band  and *reliable* of course!

All the more so since anyway I have already a session to setup
with several round trips and a SDP to exchange etc ...

How can it be better to do it in band ?

regards

Philippe Gentric
Software Architect
Philips MP4Net

                                                                                                                                       
                                                   To:   Philippe Gentric/MP4-SUR/CE/PHILIPS@EMEA1                                     
                                                    "Even, Roni" <[email-masked]>                                             
                                                   cc:   [email-masked]                                                                  
                                                    "'Stephan Wenger'" <[email-masked]>                                         
               "Even, Roni"                        Subject:    RE: [AVT] draft-ietf-avt-rtp-h264-01.txt in band PS                     
               <[email-masked]                                                                                                
               >                                   Classification:                                                                     
                                                                                                                                       
               13/03/2003 16:15                                                                                                        

Hi Phillippe,

Regards
Roni Even

---

From: [email-masked]
Date: Thu, 13 Mar 2003 16:33:50 +0100

Roni,

you write:

Hum.

You imply that the server should send all the "required" Parameters Sets,
it would be better if it was explicit.

anyway this does not solve my problem which is that I dont want
to be forced to implement it in my VOD servers and clients
and I prefer some clear normative text for it!

regards,

Philippe Gentric
Software Architect
Philips MP4Net

                                                                                                                                       
                                                                                                                                       
               13/03/2003 16:15                                                                                                        

Hi Phillippe,

Regards
Roni Even

---

From: "Even, Roni" <[email-masked]>
Date: Thu, 13 Mar 2003 17:48:30 +0200

Phillipe,
In your case it may be better to use reliable protocol to send the parameter
set. The problem is with different applications that may switch the RTP
source during the call. Sending all the parameter sets in the beginning may
be difficult since each source may want to use the same parameter set
numbers with different values so some mechanism to allocate the numbers is
needed. The other option is to use out of band signalling with each switch
of source. If you use out of band in this case it will be long and you will
need to synchronize so that the sender will not send before the receiver
arrives. Using in band non reliable has lower latency and losing the PS is
the same as losing the full picture that you need from the new source so you
need to protect both if error resilience is a concern

Roni

> Sent: Thursday, March 13, 2003 5:25 PM
> To: Even, Roni
> Philippe Gentric/MP4-SUR/CE/PHILIPS@EMEA1                     
>                 
>                                                     "Even, 
>                "Even, Roni"                        Subject:   
>  RE: [AVT] draft-ietf-avt-rtp-h264-01.txt in band PS          
>            
>                <[email-masked]                       
>                                                               
>            
>                >                                   
> Classification:                                               
>                       
>                                                               
>                                                               
>            
>                13/03/2003 16:15                               
>                                                               
>            
>                                                               
>                                                               
>            
>                                                               
>                                                               
>            
> 
> 
> 
> 
> Hi Phillippe,
>