Subject: [AVT] H.264 payload spec: a few remarks

From: [email-masked]
Date: Tue, 11 Mar 2003 14:09:20 +0100

stephen, all,

a few remarks on H264 RTP payload:

********************

      General remarks:

   The title(!) and the text keeps on using "JVT" while the file
   name, the abtract and introduction use the term H.264. (and
   introduction explains why H.264 is the correct name!) I think a
   global find-and-replace(JVT->H.264) is needed!

   The term "NALU" is defined but after that "NAL unit" is also
   used, consistency could help.

There are quite a few reference problems i.e.
      * figures and table numbers
      * "above" used for things that are below and vice versa
      * wrong internal section references

   Some acronyms are not explained: SEI, IDC (also "Idc"?), DPA,
   DPB, DPC (even FEC etc), (could be more I did not check), also I
   am always fascinated by recursivity in acronyms: "NRI: NAL
   Reference IDC" (this one is superb!-)

      Detailed remarks:

      Section 1.1

   I suspect the sentence "Neither VCL nor NAL are claimed to be
   media or network independent" should be removed because it is not
   logically correlated to the rest of the paragraph.

   About the paragraph:

   "Also, there are NAL units that are affecting many pictures and
   are, hence, inherently time-less. For this reason, the handling
   of the RTP timestamp requires some special considerations for
   those NALUs for which the sampling or presentation time is not
   defined, or, at transmission time, unknown."

   Describing these NALU as being "time-less" and of having
   "presentation time [] not defined" could cause some mis-
   understanding, another formulation could be:

   "Some NALU transport information that affect the decoding process
   over a period of time corresponding to many video frames; For
   these units, the handling of the RTP timestamp requires some
   special considerations especially in cases where this duration is
   unknown at transmission time." BTW the same remark applies in section 6.1 for "If the NALU has
   no own timing properties". You should also elaborate i.e. start
   to present _how_ the "special considerations" are implemented in
   the spec ...

   About the sentence:

   "The combination of the higher level parameters is called a
   Parameter Set".

   It would be useful to add:

   "which is a data structure akin to what is described as "decoder
   configuration" in MPEG-4 specifications."

   since this concept is quite specific to MPEG-4 codecs and H.264
   (e.g. most speech or "older" codecs do not have this etc).

   About the sentence:

   "Although, conceptually, the Parameter Set updates are not
   designed to be sent in the synchronous packet stream, this memo
   contains means to convey them in the RTP packet stream."
   It is strange (actually potentially contradictory: if they are not
   "designed" to be sent in packets you would need to document how
   to map them into NALU but then why the draft does not do that?),
   perhaps you should replace "designed" with "intended"? in any
   case it could benefit a little explanation of the "why".

   The sentence:

   "The JVT specification declares a value of 1 as a syntax
   violation"

   I like this old joke ... ah syntax violation ... is it a
   disgusting crime or something one might indulge oneself in?-)
   but at first it _really_ reads as "it is forbiden to set this bit to
   1" so your reader will start scratching his (her) head... what
   about:

   "The F bit when set to 1 signals that the NALU or header contains
   a syntax error, in the context of this specification it may have
   been set by a network device upon detection that a transmission
   error has affected the corresponding data. A prudent reaction of
   decoders that are incapable of handling bit errors is to discard
   such packets. "

   In the "4. scope" section one finds:

   "Likely, the first applications of a Standard Track RFC resulting
   from this draft will be in the conversational multimedia field,
   video telephone or video conference"

   This is all very good but something is missing there. I think the
   problem is not what will be the "first" application but rather
   what is the _intention_ and _suitability_. This section does not
   say anything about the suitability of the specification for say
   broadcast or video on demand, which is at minimum extremely
   surprising since in the introduction the reader was entertained
   about "a very broad application range" and how good H264 is to
   transport satellite quality at 1.5 Mb/s!

   In section 6.3.1

   I could not find _how_ DON is computed except "The DON of the
   first NALU in transmission order MAY be set to any value", if you
   dont mandate anything it should be explicit for example "It is
   not required that DON are monotonously incremented"

   in the figure "NALU payload format for STAP." (same for the next
   2 figures) what are the first 8 bits ?

   Sometimes one finds "NALU type octet" some other times "|F|NRI|
   type   |"; consistency would help.

   By "D1 . D2 >= 32768" you probably mean "D1 - D2 >= 32768" ?

   It says: "NAL units associated with different primary coded
   pictures according to subclause 7.4.1.2 of [1] MUST NOT have the
   same value of DON." Is it possible to have an accidental
   collision? Then what could be the consequence?

   In section 6.3.2

   "DON of the following NALU is equal to DONB + DOND and MUST NOT
   be larger than 65535" You mean that DON=DONB+DOND MUST NOT roll
   over inside a packet, right? It would help to add if DONB roll
   over can occur at any value (also see above "not monotonously
   incremented" rule?) for the next packet...

   In Section 6.4. "Fragmentation Units " I could not find the
   indication that there cannot be more than one FU per RTP packet?
   I think it is the case? if it is  it should be more explicit.
   (and also, why then call it a "unit" rather than a "packet" ?)

   "R: 1 bit The Reserved bit MUST be 0." sheer curiosity: what is
   the purpose of a reserved bit with a _forever_ fixed value ?

   "A FU payload can have any number of octets and can be empty" why
   (what for?) can it be empty?

   In Section 7.1 about the sentence:

   "Some receivers MAY NOT support a transmission order that does
   not conform to the NAL unit decoding order. "

   By what mean the sender can find out about that?

   In Section 10. Security Considerations:

   What about crashing a decoder by sending a lethal Parameter Set ?

   References:

   for [14] I found it online at: http://www.diee.unica.it/pv2000/proceedings/papers/30.pdf

   The link http://www.pv2002.org is broken(?) but I found that paper at:
   http://www-video.eecs.berkeley.edu/papers/sangk7/pv2002.doc

************************************************************

regards,

Philippe Gentric
Software Architect
Philips MP4Net
[email-masked]
http://www.platform4.philips.com

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: Stephan Wenger <[email-masked]>
Date: Wed, 12 Mar 2003 05:45:53 +0700

Folks,

At 02:09 PM 3/11/2003 +0100, [email-masked] wrote:
>[...]
>    The title(!) and the text keeps on using "JVT" while the file name, 
> the abtract and introduction use the term H.264. [...].  I think a

or a change back to JVT, if MPEG endorses this draft as a way to transport 
this new video bit stream.  The inconsistency will be fixed, but IMHO it 
was not worth to spend the time before the I-D deadline.

>    The term "NALU" is defined but after that "NAL unit" is also used, 
> consistency could help.

Agreed, will fix

>       * wrong internal section references

Agreed, will fix

>Some acronyms are not explained: SEI, IDC (also "Idc"?), DPA, DPB, DPC 
>(even FEC etc), (could be more I did not check), also I am always 
>fascinated by recursivity in acronyms: "NRI: NAL Reference IDC" (this one 
>is superb!-)

All these are either taken verbatim from the definitions section or from 
the syntax/semantic description of the H.264 spec.  Clearly, the payload 
format will not make much sense to anyone who doesn't have the payload spec 
itself in his/her hands, but isn't this true for all payload 
formats?  Nevertheless, we may add some non-normative, explanatory text 
(don't know yet how much -- is a minor issue).

With respect to the rest of this Email: A few points can be answered 
directly, without much discussions between the authors.  On others I need 
to consult first.  I removed the latter from the EMail to save space and 
reading time, and replaced them by [...].

[...]
>       Section 1.1
>[...]   Describing these NALU as being "time-less" and of having

Here in Pattaya we decided a few hours ago that, for the purpose of the 
video buffer management (compliance tests and such, has nothing to do with 
RTP), all NAL units have a picture associated with and hence carry time.  I 
heard that the MPEG systems folk use the same type of 
definition.  Following this course would be the easy way out.
However, there are NALU types which -- I continue to believe -- still 
inherently timeless.  The two parameter set NALU types, for example, are, 
once they are transmitted and unless they are overwritten, valid for the 
lifetime of the session.  We will try to add clarifying remarks to this 
section.

>    "The combination of the higher level parameters is called a Parameter 
> Set".

Not sure whether they are really "akin" (I tend to believe they are not, 
but I will talk to people here and check whether such a comparison is 
valid.  I also tend to believe that the MPEG-4 specs are not that well 
known that this remark would benefit a large user community, while it could 
guide others on the wrong path.  Finally, I would like to avoid normatively 
referencing other MPEG documents but 14496 Part 10 (as I try to avoid 
mentioning ITU Recs such as H.245, which could be used to carry Parameter 
Sets as well and has a similar concept).

>[...]   "Although, conceptually, the Parameter Set updates are not 
>designed to be sent in the synchronous packet stream, this memo contains 
>means to convey them in the RTP packet stream."  It is strange (actually 
>potentially contradictory: if they are not "designed" to be sent in 
>packets you would need to document how to map them into NALU but then why 
>the draft does not do that?) [...]

Agreed to fix.  Don't know how, yet, but this language is strange, indeed.

>[...]
>"The F bit when set to 1 signals that the NALU or header contains a syntax 
>error, in the context of this specification it may have been set by a 
>network device upon detection that a transmission error has affected the 
>corresponding data. A prudent reaction of decoders that are incapable of 
>handling bit errors is to discard such packets. "

This language, while technically correct, is politically incorrect (at 
least for the JVT spec itself).  I think what we have is good enough, but 
we will check whether informative language along the lines of your 
suggestion are possible.

[...]

>    In section 6.3.1 [...]

Miska and I are working on the complete DON and FU sections (6.3 and 6.4) 
here in Pattaya.  I agree that both sections need some serious work.

>"Some receivers MAY NOT support a transmission order that does not conform 
>to the NAL unit decoding order. "

unspecified control protocol means, or implicit (application dependent).  I 
think this is mentioned, but we will double-check.

>    What about crashing a decoder by sending a lethal Parameter Set ?

Or sending a lethal other NAL unit, e.g. an SEI that sets the video buffer 
size to 1 byte, or, or, or.  There are many ways to kill a video 
codec.  But I agree that there is work needed in the security section re 
asymmetric operation characteristics and user data.  I hope to make some 
progress on that in San Francisco

>    References:
>    for [14] I found it online at: 
> http://www.diee.unica.it/pv2000/proceedings/papers/30.pdf

Thanks, will be added.

I want to thank you (and Magnus) again for your careful 
proof-reading.  This helps a lot.

Stephan