Subject: Re: [AVT] New I-D on JVT/H.26L packetization

From: [email-masked]
Date: Tue, 5 Mar 2002 14:43:50 +0100

Stephan,

a few remarks about your draft:

*****************
 I dont see the lack of final status for NAL as a problem, on the contrary
I beleive that making sure that the (RTP) transport is independent of 
the NAL internals is a key design issue.

in this respect, you state that:

"The RTP payload specification is designed to be 
   unaware of the bit string in the NALP payload."

but you dont state explicitely what is the situation for NALP headers,
rather you write:

 "JVT  Video-aware network elements such as Gateways can perform many 
   operations by handling only those byte buffers"

but is this in the scope of a RTP payload format ?

 I dont think so, being able to re-packetize is a *extremely* nice 
property of JVT/NAL. 

Fine, but why would you specify that in the RTP payload ?
same for the NALP type byte, why this table of value?

****************** 
 I am -as you are- concerned about using the RFC2250 timestamp policy.

You write: "Clearly, using the RFC2250-like timestamp disallows the exact 
media 
   synchronization between different RTP receivers"

if you mean not being able to synchronise with other RTP streams
then this is simply not acceptable.

Also I dont understand the logic of the choice, specifically the 
statement:

"A consequence of this new feature (it was available 
   before only in H.263++ [3]) is the complete decoupling of the 
   transmission time, the decoding time, and the sampling or 
   presentation time of slices and pictures"

seams a bit extreme to me...

Another way to look at this issue is to consider that these video codecs
behave like audio codecs with an arbitrarily complex interleaving 
scheme...

moreover each fragment (slice) objectively HAS a presentation time stamp
which is the CTS (composition time stamp) of the corresponding Access Unit 
(picture)...

so what you need is an interleaving scheme that can re-order the fragments
and assign CTS to each ... for which I know a method ;-)

the fact that you want to send fragments a long time before they are to be 
used
is a separate (buffer management, etc) issue ...

BTW the description of the example of usage of ERPS in 10.1 would need
some more details in terms of rationale (why exactly you want to spread
the I frame over 10 minutes is unclear)

regards,

Philippe Gentric
Software Architect
Philips MP4Net
[email-masked]
http://www.mpeg-4.philips.com

Stephan Wenger <[email-masked]>
Sent by: [email-masked]
02/24/2002 18:27

        To:     [email-masked]
        cc:     thomas Stockhammer <[email-masked]>
[email-masked]
(bcc: Philippe Gentric/LIM/CE/PHILIPS)
        Subject:        [AVT] New I-D on JVT/H.26L packetization
        Classification: 

Folks,

My new draft for an RTP packetization scheme for JVT video (aka H.26L, aka 

forthcoming MPEG-4 Part 10) is now available from the I-D archives as 
ftp://ftp.ietf.org/internet-drafts/draft-wenger-avt-rtp-jvt-00.txt.

This draft is under development by Tom, Miska, and myself since quite some 

time.  It is submitted by the authors, and not on behalf of the whole JVT 
group, as there are people in this group who would prefer to see an 
approach that is more aligned with other payload specifications, 
particularly with the MPEG-4 multisl draft.

While we have had more than 10 turnaround between us authors, this is 
still 
a true -00 draft, with a lot of known (and undoubtly many unknown) 
problems.  Any help on spotting and solving such problems is, as usual, 
appreciated.  Also, the video coding standard, and particularly the 
Network 
Adpatation Layer is not yet set in stone, and we hope to receive valuable 
input at the AVT meeting to make the video coding itself more network 
friendly.

One of the key problems of the development (and the review) of this draft 
is that JVT doesn't have a complete, accurate, and up-to-date description 
of the video syntax, or, more precisely, the syntax of the Network 
Adaptation Layer.  A document describing this syntax and the rationales 
behind it is currently in preparation, and I will advise you on this 
mailing list as soon as it is available.  For now, the references in the 
draft will have to suffice.

Best regards
Stephan

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 07 Mar 2002 17:31:54 +0100

Hi Philippe,

At 11:48 AM 3/7/2002 +0100, you wrote:
>dont you agree that a "perfect" media codec would be one that
>would provide "packets" that you can simply *fit* into one RTP
>packet as payload ?

I prefer to solve problems where they occur, and not try to put
any and all mechanisms for any and all networks (including
those not yet designed) into the video coding.  This, again, is
the very reason for the existence of RTP payload specs -- they
solve the problem of adapting a media stream to one specific
network/protocol environment.

One could argue that the NAL has the same task, and I agree.
I don't really care, where precisely we put the mechanisms
at the end of the day.  In fact, currently, the H.26L draft
currently does not contain a lot of mechanisms itself, but
rather informal descriptions of mechanisms specified in the NAL.

However, there are things about video that the protocol implementer
has to be aware of.  You can't hide them all.  It was tried, and the
approach failed.

>with H26L (and NAL) we have the opportunity to do that since
>we are designing the bit stream syntax, the NAL and the RTP payload format
>*at the same time*.

It is my hope that we come up with a consistent design.  That is why
this draft is presented so exceptionally early.  But just saying we move
all functionality necessary for the decent support of RTP to the JVT
group and present a draft along your lines would disallow us to use
the intelligence of the AVT group.  Here is the core experience for
RTP packetization, and not in JVT.  Not to speak of the mandate :-)

My two cents,
Stephan

---

From: [email-masked]
Date: Fri, 8 Mar 2002 17:39:44 +0100

The fact that (traditionally) video codecs maintain time in the bits tream
(you quoted H263, MPEG-4 video is similar) is indeed a source of bad 
difficulties.

It is my understanding that H26L would do the same thing (?)

One not documented approach for MPEG-4 video seems to be that
a player should rather ignore the time in the bit stream (?)

Regards,

Philippe Gentric
Software Architect
Philips MP4Net

Magnus Westerlund <[email-masked]>
03/07/2002 12:58

        To:     Colin Perkins <[email-masked]>
        cc:     Stephan Wenger <[email-masked]>
Philippe Gentric/LIM/CE/PHILIPS@EMEA1
[email-masked]
[email-masked]
thomas Stockhammer <[email-masked]>
        Subject:        Re: [AVT] New I-D on JVT/H.26L packetization
        Classification: 

Colin,

I think there is need for some clarification on your comment.

>>The time stamp issue is a very difficult topic.  Plain fact is:
>>
>>a) in H.26L you don't necessarily know the time at which the content
>>of a certain packet is displayed (my understanding of the word
>>presentation timestamp).  It is possible to send data that gets
>>decoded, but gets NEVER displayed.  And I can show you scenarios
>>where such a (at the first glance weird) scheme is actually useful.
>>
>
>That shouldn't be an issue, since RTP makes no comment on when the media
>data is displayed. We do need to figure out how the RTP timestamp relates
>to the (various?) H.26L timelines.
>

You are saying that RTP makes no comment on when the media is displayed, 
my interpretation of that sentence is: As RTP timestamp represents the 
sampling instant it does not explicit express a display time, however the 
display time is often implicit given by the RTP timestamp. Otherwise your 
comment can easily be miss interpreted by people not knowing RTP very 
well.

I agree that it is important to decide how to map the H.26L timeline to 
the RTP timestamp. I believe it is also important to avoid ambiguities, 
like duplicating timeline information in both RTP header and bit stream. 
This has happened in H.263 and creates problems when for example a RTSP 
server pause a video stream for 5 seconds and then resume. In that case 
the RTP timestamp will increase by 5 seconds but the temporal reference in 
the bit stream does not reflect this. 

Regards 

Magnus Westerlund 

Multimedia Technologies, Ericsson Research ERA/T/VA
----------------------------------------------------------------------
Ericsson Radio Systems AB  | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

---

From: Stephan Wenger <[email-masked]>
Date: Fri, 08 Mar 2002 18:38:19 +0100

Hi Philippe,

>It is my understanding that H26L would do the same thing (?)

At the moment, the H.26L WD uses a thing called Picture ID which
is necessary for the reference buffer handling and also abused as a TR
in the H.261 sense.  However, we currently don't have concepts such
as a setting of the picture clock frequency that would make the TR
protperty of the PictureID useful.

There will be a proposal from me (and likely some more people) to the
Fairfax meeting to allow the transmission of a presentation time stamp
in the form of an SEI message.  Applications which cannot rely on
a mux such as the MPEG mux, can use this.  Note that one of this
applications may be to carry H.26L over RTP, using the draft in question,
without having additional timing info available in the form of a non-RTP
mux mechanism.

But I agree 100% with you that there is no point in always sending timing
information in every H.26L stream.

Stephan