Subject: RE: [AVT] Updated issues to discuss -- draft-ietf-avt-rtp-h264-01 .txt

From: "Even, Roni" <[email-masked]>
Date: Thu, 13 Mar 2003 12:22:11 +0200

Hi,
I would like to point out that there are problems with supporting parameters
sets in multipoint video conferencing scenarios. When using out of band
signalling for the parameter sets if a video switch need to occur then there
is a need for a new parameter set for the new video stream and it has to be
available to the receiver before the actual stream.
The current work at the ITU suggests to use in band parameter sets and to
send the parameter sets before the full picture as a response to a fast
update request. (available in the RTCP feedback draft)
Regards
Roni Even

> -----Original Message-----
> From: Stephan Wenger [mailto:[email-masked]]
> Sent: Thursday, March 13, 2003 11:38 AM
> To: [email-masked]
> Subject: [AVT] Updated issues to discuss -- 
> draft-ietf-avt-rtp-h264-01.txt
> 
> 
> Folks,
> here is an updated list of the issues re the H.264/JVT 
> packetization.  It 
> reflects mostly the discussions here in Pattaya, where MPEG and JVT 
> currently meets:
> 
> Unchanged items:
> 1. Status of JVT -- should be out for the final ballot by the 
> time of the 
> meeting.
> 2. Fragmentation -- we decided on adding application layer 
> fragmentation of 
> NALUs to support some exotic scenarios.  I plan to describe 
> the pros and 
> cons of the fragmentation, and hope for more guidance from 
> the meeting re 
> these feature.  We had discussed this briefly in Atlanta 
> already, but it 
> has turned out that I forgot some applications where 
> fragmentation is useful.
> 3. MIME/SDP sections: are they ok?
> 4. Interaction with MPEG-4 simple packetization (since H.264 is also 
> ISO/IEC 14496-10).  Miska and I are currently in Pattaya at 
> the JVT/MPEG 
> meeting, as are Jan Van-der-Meer, Dave Singer, Young-Kown Lim 
> (and other 
> knowledgeable people).  We will have some discussions about 
> the interaction 
> of the two drafts, and report probably to the mailing list, 
> but certainly 
> orally at the meeting, about the outcome of these discussions.
> 5. I currently believe that the problem mentioned in section 
> 12 of the 
> draft doesn't exist, but we will discuss this here in Pattaya 
> as well and 
> take action as necessary.
> 6. Security section seems to need more work, but this is not 
> reflected in 
> the draft yet.  We plan to present a summary of the problems 
> and proposed 
> solution if there is time left.
> 
> New items:
> 7. Interoperability between MPEG-4 simple and H.264 
> packetization -- as it 
> has turned out, we already have one common operation point.  
> Now that AUs 
> are syntactically defined in MPEG-4, we can add something 
> similar to a STAP 
> that adds another common operation point.  Our goal is to 
> make this draft a 
> binary-compatible superset of the MP4-Simple draft when 
> carrying H.264, but 
> it is unclear whether this can be achieved.
> 8. MIME/SDP codepoints to support out-of-level picture sizes 
> -- the video 
> conferencing industry needs to support extreme picture sizes 
> (e.g. 1500 x 
> 1000 pels) at low frame rates, for example for document 
> cameras.  However, 
> they don't want to signal a profile/level combination that 
> allows for this 
> picture size (memory requirements), because that also implies 
> the ability 
> to decode high speed bit streams at high frame rates (computational 
> requirements).  Such codepoints are in principle agreed in 
> the ITU-T H.245 
> protocol, and we are asked to add similar codepoints in the MIME/SDP.
> 9. I will be prepared to explain DON (why it is there, and it 
> works), but 
> we admit that there is an urgent need for clearer language for both.
> 
> Regards,
> Stephan
> 
> _______________________________________________
> Audio/Video Transport Working Group
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/avt
> 
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 13 Mar 2003 18:49:11 +0700

Hi Roni,

At 12:22 PM 3/13/2003 +0200, Even, Roni wrote:
>[...]
>I would like to point out that there are problems with supporting 
>parameters sets in multipoint video conferencing scenarios.
>[...]

First, please let me point out that Roni's multipoint is not the multipoint 
most of the AVT'ers would think of.  Roni speaks, I believe, primarily 
about a multipoint scenario where a central Multipoint Control Unit (MCU) 
is employed.  For those who don't know, an MCU is something similar to, but 
not quite the same as, an RTP Mixer.  It also allows for stream switching 
in the same RTP session, in addition to spatial mixing (actually, there are 
product that do switching only).

Second, I agree that there are issues with parameter sets in these 
scenarios, and I know that people in the ITU are working on those.  I am 
one of them.  If the outcome of the ITU discussions is that they will use 
in-band parameter sets over IP links, then there is indeed a need to send 
parameter sets closely aligned to the IDR pictures.  Hence, technically, 
Roni and me are in a perfect technical agreement.

However, the question is whether the RTP payload specification is the right 
place to express the mentioned need.  There are many totally valid 
examples, where RTP-based video, even when using in-band parameter sets, 
will NOT require the parameter sets being sent with the IDR picture.  And I 
think I could also come up with at least one example where it would 
actually be harmful to do so.

Also, I haven't seen similar statements in RFC2429, where the analog 
language would be that you have to send the full picture header with all 
it's extensions with every I picture (or, at least, after every Full Intra 
Request or something).

We may add an informal comment on the close relationship between parameter 
sets and IDR pictures in H.264, but more seems to be 
inappropriate.  However, I would be more than happy to learn what draft/RFC 
would be the right place (in the ITU world, IMHO the right place would be 
the system overview spec, e.g. H.320 or H.323).

Regards,
Stephan