Subject: Re: [AVT] I-D ACTION:draft-ietf-avt-uncomp-video-02.txt

From: [email-masked]
Date: Thu, 6 Mar 2003 11:45:54 +0100

Hi all,

I concur with chuck, this is looking very good indeed.

Just a few remarks

*****************************
I am not sure I always understand the "pixels" and "#words" second and third column in the tables ?

For exemple for monochrome 10 bits pixels is "4" so that would hint it means "number of pixels per Pgroup"

But then the next line is "4" for 4:1:1 while there are 8 pixels per Pgroup?

Also the #words is 6x10, but these are actually "bits" rather than "words"?

I think these tables would benefit a few sentences giving the exact definition of each column.

*****************************
The 8 bit words tables probably contains a typo for the "octet alignement" of Interlaced 4:2:0 which should be "4x8/8 = 4"
no?

*****************************

in section 4.3 it says

"For RGB format video, samples are packed in order Red-Green-Blue.  All
samples are the same bit size, which may be 8, 10, 12, or 16 bits.  If 8
bit samples are used, the pgroup is 3 octets. If 10 bit samples are
used, samples from adjacent pixels are packed with no padding, and the
pgroup is 15 octets (4 pixels).  Refer to Tables 1 thru 4.

For RGBA format video, samples are packed in order Red-Green-Blue-Alpha.
All samples are the same bit size, which may be 8, 10, 12, or 16 bits.
For pgroups refer to Tables 1 thru 4."

Unfortunately the tables 1 thru 4 do not explicitely mention any RGB or RGBA format(!)

I think it would be useful to add them, here are the tables
(not sure the notation R:G:B is appropriate, anyway  "RGB" is
what is declared in the mime type ...)

                                                 8 bit words
          Color                --------------------------------------
      Subsampling Pixels       #words octet alignment samples  pgroup
                                                               octets
     +-----------+------+---+ +------+---------------+---------------+
     |   R:G:B   |  1   | P | | 3x8  |  3x8/8 = 3    |   3   |   3   |
     +-----------+------+---+ +------+---------------+---------------+
     |  R:G:B:A  |  1   | P | | 4x8  |  4x8/8 = 4    |   4   |   4   |
     +-----------+------+---+ +------+---------------+---------------+

                                               10 bit words
          Color                --------------------------------------
      Subsampling Pixels       #words octet alignment samples  pgroup
                                                               octets
     +-----------+------+---+ +------+---------------+---------------+
     |   R:G:B   |  1   | P | | 3x10 |  4x30/8 = 15  |   12   | 15   |
     +-----------+------+---+ +------+---------------+---------------+
     |  R:G:B:A  |  1   | P | | 4x10 |    40/8 = 5   |    4   |  5   |
     +-----------+------+---+ +------+---------------+---------------+

                                            12 bit words
          Color                --------------------------------------
      Subsampling Pixels       #words octet alignment samples  pgroup
                                                               octets
     +-----------+------+---+ +------+---------------+---------------+
     |   R:G:B   |  1   | P | | 6x12 |  2x36/8 = 9   |    6   |  9   |
     +-----------+------+---+ +------+---------------+---------------+
     |  R:G:B:A  |  1   | P | | 4x12 |    48/8 = 6   |    4   |  6   |
     +-----------+------+---+ +------+---------------+---------------+

                                             16 bit words
          Color                --------------------------------------
      Subsampling Pixels       #words octet alignment samples  pgroup
                                                               octets
     +-----------+------+---+ +------+---------------+-------+-------+
     |   R:G:B   |  1   | P | | 3x16 | 3x16/8 = 6    |   3   |   6   |
     +-----------+------+---+ +------+---------------+-------+-------+
     |  R:G:B:A  |  1   | P | | 4x16 | 4x16/8 = 8    |   4   |   8   |
     +-----------+------+---+ +------+---------------+-------+-------+

*****************************

Since we have 4:4:4:4, one thing that should be added is 4:4:2:0
which is 4:2:0 (the most common raw video format
since it is the raw format of most codecs) plus
a transparency plane with one transparency word
per pixel i.e. same amount of transparency as luminance

This can be *very* useful for studio applications and is also
directly compressable using the profiles and levels of
MPEG-4 video that support the "shape" tool.

It simply needs to add 4:4:2:0 in the valid value list (section 6.1)
and in section 3 the relevant tables would be:

                                                 8 bit words
          Color                --------------------------------------
      Subsampling Pixels       #words octet alignment samples  pgroup
                                                               octets
     +-----------+------+---+ +------+---------------+---------------+
     |  4:4:2:0  |  4   | P | | 10x8 |  80/8 = 10    |   10  |   10  |
     +-----------+------+---+ +------+---------------+---------------+
     |  4:4:2:0  |  4   | I | | 8x8  |  64/8 = 8     |   8   |   8   |
     +-----------+------+---+ +------+---------------+---------------+

                                                10 bit words
          Color                --------------------------------------
      Subsampling Pixels       #words octet alignment samples  pgroup
                                                               octets
     +-----------+------+---+ +------+---------------+---------------+
     |  4:4:2:0  |  4   | P | | 10x10| 2x100/8 = 25  |   20  |   25  |
     +-----------+------+---+ +------+---------------+---------------+
     |  4:4:2:0  |  4   | I | | 8x10 |   80/8 = 10   |   8   |   10  |
     +-----------+------+---+ +------+---------------+---------------+

                                                 12 bit words
          Color                --------------------------------------
      Subsampling Pixels       #words octet alignment samples  pgroup
                                                               octets
     +-----------+------+---+ +------+---------------+---------------+
     |  4:4:2:0  |  4   | P | | 10x12|   120/8 = 15  |   10  |   15  |
     +-----------+------+---+ +------+---------------+---------------+
     |  4:4:2:0  |  4   | I | | 8x12 |   96/8 = 12   |   8   |   12  |
     +-----------+------+---+ +------+---------------+---------------+

                                                 16 bit words
          Color                --------------------------------------
      Subsampling Pixels       #words octet alignment samples  pgroup
                                                               octets
     +-----------+------+---+ +------+---------------+---------------+
     |  4:4:2:0  |  4   | P | | 10x16|   160/8 = 20  |   10  |   20  |
     +-----------+------+---+ +------+---------------+---------------+
     |  4:4:2:0  |  4   | I | | 8x16 |  128/8 = 16   |   8   |   16  |
     +-----------+------+---+ +------+---------------+---------------+

regards,

Philippe Gentric
Software Architect
Philips MP4Net
[email-masked]
http://www.platform4.philips.com

                                                                                                                                       
                                                   To:   [email-masked]                                                                  
                                                    Ladan Gharai <[email-masked]>                                                  
                                                   cc:   (bcc: Philippe Gentric/MP4-SUR/CE/PHILIPS)                                    
                                                   Subject:    Re: [AVT] I-D ACTION:draft-ietf-avt-uncomp-video-02.txt                 
               Chuck Harrison                                                                                                          
               <[email-masked]>                  Classification:                                                                     
                                                                                                                                       
               Sent by:                                                                                                                
               [email-masked]                                                                                                      
                                                                                                                                       
               05/03/2003 19:19                                                                                                        

Hi, all,

A few comments on this draft, which is looking very good so far.

---
In uncomp-video-02 we have:

4.3.  Payload Data

Depending on the video format, each RTP packet can include either a
single complete scan line, a single fragment of a scan line, or one
(or
more) complete scan lines plus a fragment of a scan line. [...]

I think there are additional logical possibilities allowed by
the document as it stands: a packet may begin with the tail
fragment of a line, then continue with 0 or more full lines, and
possibly the head fragment of another line.

If the intent is to exclude these possibilities, we need normative
language to say so. If these operational modes are permitted, the
quoted text above is misleading and should be amended.

---
On page 3 of the draft, we have

[...] This payload format
requires that applications using such synchronized ancillary data MUST
deliver it in separate RTP sessions which operate concurrently with
the
video session.  The normal RTP mechanisms SHOULD be used to
synchronize
the media.

I believe the MUST is inappropriate and SHOULD is preferable.

The idea that frame-accurate timecode will be carried as a
parallel RTP session is a laudable but forward-looking conjecture.
IMHO reliable use of this payload format in the near future
will depend on carriage of vertical interval timecode (VITC)
in an unused scan line which is part of the vertical blanking
interval. (This is normal video operational practice, as
mentioned in RFC 2431.)

At the top of page 2 of the draft, we have "...the contents of
horizontal and vertical blanking SHOULD NOT be transported." This
does not bother me as SHOULD NOT language does not prevent me from
carrying VITC when I need it, until the multisession RTP sync
issues get ironed out. On the other hand the MUST language of
page 3 is burdensome.

---
<nitpick>
A minor MUST/SHOULD nit at sec. 4.3 regarding filling to a pgroup
boundary: since a receiver MUST ignore the fill data, it seems
unneccessary that the sender MUST fill with zeroes, specifically.
The sender MUST fill, and SHOULD fill with zeroes.
</nitpick>

---

The MIME and SDP parameters do not include the video frame rate.
There are many plausible scenarios in which it is important
to distinguish alternative frame rates. For example, choosing
the right VOD stream during RTSP setup would need something like
this.

---

Cheers,
  Chuck Harrison
  Far Field Associates, LLC
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: Chuck Harrison <[email-masked]>
Date: Fri, 07 Mar 2003 12:18:09 -0800

Philippe, all,

> I concur with chuck, this is looking very good indeed.
[...]
> what is declared in the mime type ...) [...]

I had assumed that the conjunction of color-mode = RGB and 
sub-sampling = 4:4:4 would describe this. Thus the tables 1 thru
4 would not require additional entries distinguishing one color-
mode from another.

As you say, the tables could use a legend/explanation. I think
this could be clarified there.

While we're on color specs, how about this, to encourage *complete*
color specifications:

----
6.2.  Parameter Registration

New values of the "colorimetry" parameter MAY be registered with the
IANA provided they reference an RFC or other permanent and readily
available specification (the Specification Required policy of RFC 2434
[2434]). <New text:> New registrations SHOULD define all parameters
of a color image encoding as described in [22028].
----

Note: ISO 22028-1 is still in ballot with TC42 for another few days
but I am hopeful that it will be adopted by the time this internet-
draft reaches IESG.

If anyone is interested in the contents of 22028-1 please contact
me off list.

Cheers,
  Chuck Harrison

---

From: Ladan Gharai <[email-masked]>
Date: Fri, 07 Mar 2003 17:11:20 -0500

Hello Philippe - thank you for your comments. Please see reponses inline.

Ladan

 > I concur with chuck, this is looking very good indeed.

   Pixels refer to the smallest number of pixels a related of subsamples 
   represent, so for example for 4:4:4 Cb-Y-Cr cover 1 pixel whereas for 
   4:2:2 Cb0-Y0-Cr0-Y1 covers 2 pixels.
    
   #words, on the other hand expresses the number of words used to
   represent those pixels: 
   so for 4:4:4 requires 3 words Cb-Y-Cr and
          4:2:2 needs    4 words Cb0-Y0-Cr0-Y1

 > For exemple for monochrome 10 bits pixels is "4" so that would hint it means "number of pixels per Pgroup"

   It seems to me now that "number of pixels per Pgroup" would be a more
   intuitive way to look at pgroup - however pgroup is actually
   the number of bytes need to represent the least number of
   pixels, while being both byte aligned and observing ALF. 

   thats why, for the 10bit and 12bit tables, the two last columns
   #samples and pgroup, dont agree, whereas they agree for the 8bit table.
  
 > But then the next line is "4" for 4:1:1 while there are 8 pixels per Pgroup?

   so what that means is that the smallest group of related samples for a
   4:1:1 subsampling is 4 pixels p0 p1 p4 p3  represented as 
   Cb0-Y0-Y1-Cr0-Y2-Y3, this is 6 samples, which with 10bit words, align
   after 12 samples, or 15 octets - which is representing 8 pixels.
  
 > Also the #words is 6x10, but these are actually "bits" rather than "words"?

    true - its 6 words of 10 bits each.
  
 > I think these tables would benefit a few sentences giving the exact definition of each column.

   I couldn't agree more. The tables have acquired a life of their own. I hope my
   explanations have helped to clarify somewhat.

   yes - thank you :)

   True. I guess I was assuming that it is obvious that RGB and RGBA 
   would act the same as 4:4:4 and 4:4:4:4 subsampling.

   I wonder if it would be best to expand the tables, with the rows you
   have supplied, or add a reference in the text to the effect that 
   RGB and RGBA  pgroups are  like 4:4:4 and 4:4:4:4 pgroups? 

 > Subsampling Pixels       #words octet alignment samples  pgroup

 Yes. I shall add in values for 4:4:2:0.  Thank you.

---

From: Ladan Gharai <[email-masked]>
Date: Fri, 07 Mar 2003 17:31:52 -0500

> 4 would not require additional entries distinguishing one color-

   Well, the problem is that section 4.3 refers back to the tables,
   when discussing RGB and RGBA. A solution would be to eliminate
   the reference and instead add some text that clarifies how RGB and
   RGBA are packetized.
 
 > this could be clarified there.

   true.