Subject: Re: [AVT] Working group last call: draft-ietf-avt-mpeg4-simple-04.txt

From: [email-masked]
Date: Tue, 9 Jul 2002 17:07:19 +0200

Hello Alex,

Thanks a lot for so accurately raising issues. I numbered them in your message (attached); please find my comments below.

1) I don't think there is an unwritten rule that RTP time stamps of packets are required to monotically increase (I guess this is what you mean). In case of video with B-pictures, the pictures are indeed sent in decoding order, but the RTP time stamp
indicates the presentation time. Consequently the RTP time stamp of a P picture that is displayed subsequent to a B-picture, but that is sent prior to that B-picture has a RTP time stamp that is later than the RTP time stamp of the B-picture.  Or do I
make a mistake here ? Of course I agree that your proposal simplifies receivers, but it makes the second interleaving example in appendix A illegal.  We should be careful with such simplification, unless we would know that losing two consecutive RTP
packets is very unlikely to happen in practice.

2), 3) Agreed

4) I think your suggestion to replace "SHOULD" by "should" is OK. When multiple video pictures are concatenated in one "RFC3016 RTP packet", then the RTP receiver does not need to look for startcodes in my view; the payload contains a fully compliant
video stream which can be transferred to the video decoder without any processing by the RTP receiver.

5), 6), 7), 8), 9), 10), 11) Agreed

12) In this mode, the AU-size field is always present, also when fragmenting a single AU. I think the text should read:
   The maximum size of an AAC frame (fragment) that can be carried in an RTP packet in this mode is 8191
   octets.

13), 14) Agreed

15) I agree to follow your suggestion to remove the sentence.

16) I am not aware of any changes made  to the copyright statement. Could you please let me know what is missing ?

17) Agreed. I received a few more from John Lazarro (thanks John):
   Stuctured               --> Structured (happens a few times)
   unsufficiently adequate --> inadequate
   Lazarro                 --> Lazzaro
   Adresses                --> Addresses

Kind regards,

Jan

                      Alex MacAulay                                                                                                    
                      <[email-masked]             To:  Jan vanderMeer/EHV/BE/PHILIPS@EMEA3                                            
                      m>                            Colin Perkins <[email-masked]>                                                        
                      Sent by:                      [email-masked]                                                                       
                      [email-masked]             cc:                                                                                 
                      rg                           Subject:   Re: [AVT] Working group last call: draft-ietf-avt-mpeg4-simple-04.txt    
                                                                                                                                       
                                                   Classification:                                                                     
                      2002-07-08 09:59                                                                                                 
                      PM                                                                                                               

Hi,

I read through the -04 draft this weekend, and have a bunch of little
last-minute comments...

1) Although I know it has already been flagged elsewhere, I would like to
address the issue of the 2nd interleaving example in Appendix A. This
example presents an interleaving scheme that avoids the loss of two
consecutive audio frames in the face of the loss of two consecutive RTP
packets. It achieves this by sending RTP packets with timestamps that
are not in decoding order. Doesn't this violate a (perhaps unspoken)
rule that packets should be in decoding order, and more specifically
that the first AU of each packet should be sent in decoding order? This
unspoken rule is apparently used in the first example of section A.1,
because it says that the arrival of packet 2 allows us to notice the
loss of frame 1. The assumption is that no future packet will contain a
frame with a timestamp less than T[2] (the timestamp of packet 2), and
so we can conclude that we will never receive frame 1. However, if the
first example instead placed frame 1 in packet 3:

   Packet   Time stamp   Frame Numbers       AU-Index, AU-Index-delta
   0        T[0]         0, 3, 6             0, 2, 2
   1        T[4]         4, 7                0, 2   <--- PACKET LOST
   2        T[2]         2, 5, 8             0, 2, 2
   3        T[1]         1, 9,12,15          0, 7, 2, 2

then we would still "notice" the loss of frame 1, and emit frame 2 and
3, only to realize on the arrival of packet 3 that we did not really
lose frame 1!!! The assumption that first AUs of packets are in decoding
order is very useful for quickly detecting loss of frames. I would very
much like (as a receiver) for senders to be REQUIRED to place the first
AUs of packets in decoding order. (This is also related to the details
of the de-interleave "flushing" procedure in section 3.2.3.2).

2) In section 2.6, it says:

   The DTS may be used only in MPEG-4 video streams

However, MPEG-J streams also use the DTS for timing purposes. A simple
change to the text would be:

   The DTS may be used in MPEG-4 video streams

3) In section 2.12, it says:

   New modes can be
   defined as deemed appropriate, typically by specifications that are
   hierarchically higher than this payload format.

This sentence seems redundant and vague. Can it be removed?

4) In section 2.13, it says:

   Conversely, receivers that comply with the specification in this
   document SHOULD be able to decode payloads, names and parameters
   defined for MPEG-4 video in RFC 3016. In this respect it is

Is this a "SHOULD" in the sense that implementors of this payload should
also implement RFC 3016? Or is it really a "should" in the sense that if
you implement this payload then your implementation automatically should
be able to parse the RFC 3016 payload? I would assume the latter, in
which case "SHOULD" should be changed to "should" in the document.
However, implementations of this payload format will typically NOT have
implemented an algorithm to search for MPEG-4 video start codes in the
payload, which is necessary for receivers of RFC 3016 to extract
concatenated video frames from a single RTP packet. So I would question
whether even "should" is appropriate here.

5) In section 3.1, there is a somewhat convoluted specification for the
marker bit:

   Marker (M) bit: The M bit is set to 1 to indicate that the RTP
   packet payload includes the end of each Access Unit of which data
   is contained in this RTP packet. As the payload either carries one
   or more complete Access Units or a single fragment of an Access
   Unit, the M bit is usually set to 1, except when the packet carries
   a single fragment of an Access Unit that is not the last one.

Could this be simplified a bit? Here's an alternative:

   Marker (M) bit: The M bit is set to 1 to indicate that the RTP
   packet payload contains either the final fragment of a fragmented
   Access Unit, or one or more complete Access Units.

6) In section 3.2.1.1, the description of DTS-flag says:

         The DTS-flag field SHOULD be 0 for any non-first
         fragment of an Access Unit.

I think this may hurt packet loss recovery schemes, especially when
using MPEG-4 video with video packets. If the first fragment is lost,
then we will have lost the DTS. I would prefer the following:

         The DTS-flag field MUST have the same value for all
         fragments of an Access Unit.

And similarly for DTS-delta:

         The DTS-delta field MUST have the same value for all
         fragments of an Access Unit.

7) In section 3.2.3.2, it says:

   Access Units MAY be interleaved. Senders MAY perform interleaving.
   Receivers MUST support interleaving.

The third sentence seems to contradict the concept of "modes" because a
particular mode may restrict the usage of the payload to disallow
interleaving. For example, an implementator of a receiver for the
CELP-cbr mode may prefer not to implement de-interleaving, but is
required to do so by the above sentence. Perhaps the sentence can be
changed to:

   Receivers that fully implement this payload, as well as receivers
   that implement a mode of using this payload that allows interleaving
   (a mode that allows AU-Index to be non-zero) MUST support
   interleaving.

8) In section 3.3.3 (Constant bit-rate CELP), there is no mention of
fragmentation. Is it allowed? It would be useful to either explicitly
disallow it for this mode, or to mention that the receiver will need to
support fragmentation.

9) Similar comment for section 3.3.4 (Variable bit-rate CELP): although
it says "there is no support for fragmentation of CELP frames" it would
be preferable to say "CELP frames MUST not be fragmented when using this
mode".

10) Similar comment for section 3.3.5 (Low bit-rate AAC): it says
"optional support for interleaving and fragmenting", but I don't see an
interest in support fragmenting when the maximum frame size is 63
octects. In addition the word "optional" suggests that it is not
necessary to implement this feature, but it MUST be implemented by
receivers. Can the word "optional" be removed and fragmenting be
disallowed for this mode?

11) For section 3.3.6 (High bit-rate AAC), I would also suggest to remove
the "optional" from "optional support" to make it clear that this must
be implemented.

12) Also in section 3.3.6 (High bit-rate AAC), it says:

   The maximum size of an AAC frame (fragment) in this mode is 8191
   octets.

However, the use of the AU-size field limits the total Access Unit size
rather than the fragment size. Therefore, the sentence should say:

   The maximum size of an AAC frame in this mode is 8191 octets.

13) And again in section 3.3.6 (High bit-rate AAC), it says:

   In this mode the RTP payload consists of the AU Header Section,
   followed by one or more concatenated AAC frames.

However, this is not true in the case of fragmentation. How about
changing it to:

   In this mode the RTP payload consists of the AU Header Section,
   followed by either one AAC frame, several concatenated AAC frames,
   or one fragmented AAC frame.

14) In section 4.1, the description of the StreamType parameter says:

      streamType as defined in Table 9 (objectTypeIndication Values)

but it should be:

      streamType as defined in Table 9 (streamType Values)

15) In section 4.1, the description of the Profile parameter says:

      Absence of this parameter signals that the profile is not
      specified.

However, this does not appear to add any useful information. Would it
not be better to say:

      Absence of this parameter implies that interleaving is not used
      by the transported stream.

or to simply remove the sentence.

16) A copyright statement appears at the end of section 8. However, the
statement has apparently been cut short at some point during the editing
of the draft.

17) Finally, some spelling errors and typos that could be fixed via
search-and-replace:

    signaled --> signalled
    signaling --> signalling
    sampling instance --> sampling instant
    unsufficiently --> insufficiently
    RPT --> RTP
    haxadecimal --> hexadecimal

Alex.

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: Alex MacAulay <[email-masked]>
Date: Tue, 09 Jul 2002 19:17:51 +0200

Hi Jan,

Thanks for the quick response!

Regarding:

> 16) I am not aware of any changes made  to the copyright statement.
> Could you please let me know what is missing ?

I suspect the original copyright text may have come from RFC 2026. You
can find it there at the end of page 32 and it continues on to the
beginning of page 33.

> 1) I don't think there is an unwritten rule that RTP time stamps of
> packets are required to monotically increase (I guess this is what
> you mean). In case of video with B-pictures, the pictures are indeed
> sent in decoding order, but the RTP time stamp
> indicates the presentation time. Consequently the RTP time stamp of
> a P picture that is displayed subsequent to a B-picture, but that
> is sent prior to that B-picture has a RTP time stamp that is later
> than the RTP time stamp of the B-picture.  Or do I
> make a mistake here ? Of course I agree that your proposal simplifies
> receivers, but it makes the second interleaving example in appendix A
> illegal.  We should be careful with such simplification, unless we
> would know that losing two consecutive RTP

I agree with you that B-pictures cause the RTP time stamp to be
non-monotonic. However, the packets are still in decoding order, and
that is the unwritten rule that I wanted to express.

If we do not limit interleaving to respect this rule, then I would
assert that the first example must change. That is because when packet 2
with timestamp T[2] arrives, the example concludes that frame 1 has been
lost. This conclusion is not possible if we cannot be sure that the
first time stamps of packets are in decoding order.

Regarding the protection against the loss of two consecutive RTP
packets, what if I can convince you that we can still achieve that while
respecting decoding order of first AUs? Below is an alternative
configuration that uses continuous interleave, respects decoding order
of first AUs, and avoids the loss of two consecutive frames when two
consecutive packets are lost. (Unfortunately, it does have two
drawbacks: the first is that the first four packets in the stream are
not resilient to double-loss, the second is that the interleave delay is
larger for a given number of frames per packet).

   Packet   Time-stamp   Frame Numbers       AU-Index, AU-Index-delta 
   0        T[0]                 0   6       0  5
   1        T[1]                 1   9       0  7
   2        T[2]             2   5  12       0  2  6
   3        T[3]             3   8  15       0  4  6
   4        T[4]             4  11  18       0  6  6
   5        T[7]             7  14  21       0  6  6
   6        T[10]           10  17  24       0  6  6
   7        T[13]           13  20  27       0  6  6
   8        T[16]           16  23           0  6
   9        T[19]           19  26           0  6
   10       T[22]           22               0
   11       T[25]           25               0

Meanwhile, I have two extra comments:

- Again regarding the appendix, I think it would be nice for the
appendix to give the value of the interleave delay for each example.
This would help to ensure that sender implementations will have the
correct calculation for this value.

- It would be good to explicitly allow or disallow the repetition of an
access unit in one or more packets (i.e. two identical access units with
the same calculated AU-Index value in one or more packets). I don't
remember reading that anywhere in the document. I don't have an opinion
about whether it should be allowed or not, but I think it would be
helpful to have an explicit statement on this topic.

Alex.