Subject: [AVT] Scalable Video (JPEG-2000 & SVC)

From: "Boudani Ali" <[email-masked]>
Date: Mon, 9 Jul 2007 10:49:06 +0200

Hello,
1- In the SVC draft that describes the RTP Payload Format for SVC Video,
each RTP packet stream can carry NAL units belonging to one or more
layers. 
When multiple RTP packet streams are in use, they are session
multiplexed and form their own RTP session and therefore have their own
SSRC, PT, and Sequence numbering space. What about the timestamp (same
sampling instant)? 
Now, for the JPEG2K Codestream when transmitting layers in different RTP
sessions, do we have the same timestamp for all these packets (because
of the same sampling instant) and what about the sequence number? The
multiple RTP packet streams in use are session multiplexed, do they have
their own Sequence numbering space?.

For example what would be the values for this JPEG2K Codestream?
+----------+----+----+----+----------+----+----+----+----------+----
|Tile 0 Hdr| L0 | L1 | L2 |Tile 1 Hdr| L0 | L1 | L2 |Tile 2 Hdr| ...
+----------+----+----+----+----------+----+----+----+----------+----
RTP session 1
+----------+----+         +----------+----+        +----------+----
|Tile 0 Hdr| L0 |         |Tile 1 Hdr| L0 |        |Tile 2 Hdr| L0
+----------+----+         +----------+----+        +----------+----
RTP session 2
                +----+                    +----+
                | L1 |                    | L1 |             ....
                +----+                    +----+
RTP session 3 
                     +----+                    +----+
                     | L2 |                    | L2 |        ....
                     +----+                    +----+

2- What about using a fragment offset with SVC? The SVC draft has also
layers (like JPEG2K) but doesn't use any offset! In the other hand, the
JPEG2K draft, says that a fragment offset value MUST be set to the byte
offset of the current payload in relation to the very beginning of each
JPEG 2000 codestream (JPEG 2000 frame). Is the "MUST" is the best
keyword here. I may need it only if I am sending the different layers of
the same codestream over multiple sessions. 

Thanks,
Ali
-----Original Message-----
From: Satoshi Futemma [mailto:[email-masked]] 
Sent: lundi 28 mai 2007 09:37
To: Boudani Ali
Cc: IETF AVT WG; Colin Perkins; Andrew Leung; ITAKURA Eisaburo
Subject: Re: [AVT] Working group last call: JPEG-2000 payload format

Dear Ali,

Thank you for the comments.
Please see inline.

Boudani Ali wrote:
> Hello,
> I read the two drafts. Before going to the IESG, It will be better if
> the authors clarify some issues.
> Here are my comments:
> 
> 1- How can we identify the packets belonging to the same Codestream
> (frame) (especially when using multiple RTP sessions). Don't you think
> that we need to add a frame_id identifier?.

RTP timestamp can be used for that purpose.

> 2- In section 4.1 of Draft-ietf-avt-rtp-jpeg2000-15.txt: you consider
90
> KHz for the Timestamp, this is not enough for professional video using
> JPEG2K: isn't better to leave this to the implementers? 

RTP timestamp is a presentation timestamp.
Same frame has same rtp timestamp and timestamp difference
between frames is constant. Therefore 90KHz timestamp is enough.

How many clock rate would you consider for RTP timestamp ?
If there is a strong incentive, I feel no reluctance to
select different clock using rtpmap.

> 3- Adding a payload header is an overhead, so smaller payload header
> size is better. Now, why do we need "T" and "tile number" fields? You
> said that many tiles may exist in the same packet, so in this case no
> meaning of "tile number" and "T" fields?

As you pointed, "tile number" and "T" may not be necessary when
many tiles exists in the same packet, but if no tile header exists
in a packet, which may be more possible, "tile number" and "T"
is needed for receiver to clarify which tile the codestream
is belonging to.

> 4- Is the tp (type) field necessary?. The RTP payload has already the
> payload type (PT) and we can use it instead of tp? If not this
> information can be communicated out of band and can be optional as the
> width and height in section 8.1!

We believe it is necessary because the receiver identify whether
the received codestream is odd or even field's.

"tp" can be used for determining the reciever's action.
For example, if it receives odd field continuously, it can interpolate
even field in some way, or simply discard the duplicated odd field.

> 5- Why do you use the fragment offset?. Isn't (for one RTP session)
the
> sequence number of the RTP packet enough? 
> When using multiple RTP sessions for layered video, the parser that
> extracted the data for a particular layer (for a particular RTP
session)
> may be transmitted out of band also (the index file is stored either
as
> a separate file or as meta-data). The index file is transmitted to the
> receiver also, so the receiver can rebuild the codestream.

When using multiple RTP session for layered video,	
JPEG2000 codestream may be split with interleaving.

For example,

Codestream
+----------+----+----+----+----------+----+----+----+----------+----
|Tile 0 Hdr| L0 | L1 | L2 |Tile 1 Hdr| L0 | L1 | L2 |Tile 2 Hdr| ...
+----------+----+----+----+----------+----+----+----+----------+----

RTP 1
+----------+----+         +----------+----+        +----------+----
|Tile 0 Hdr| L0 |         |Tile 1 Hdr| L0 |        |Tile 2 Hdr| L0
+----------+----+         +----------+----+        +----------+----

RTP 2           +----+                    +----+
                | L1 |                    | L1 |             ....
                +----+                    +----+

RTP 3                +----+                    +----+
                     | L2 |                    | L2 |        ....
                     +----+                    +----+
In that case,
interleaved codestream doesn't alwasy arrive at the receiver
in original order.
Fragment offset helps the receiver to rebuild the codestream.

Also, if the fragment offset exist, the receiver need less
memory because it doesn't need de-intgerleave buffer to
rebuilt codestream. It can use same memory space to rebulit it.

> 6- Do we really need to have an MHD flag? The size of main header may
be
> about 300 bytes so is there any chance that this packet will be
> fragmented?

Typical main header may be at most 300 bytes long, but if some JPEG2000
coding options are used, it may be longer.
So MHF flag is neccessary for such situation.

> 7- For the priority field use (the example in section 1.3.1): the
> receiver can select which parts of the codestream to decode without
> using the priority field. It can rebuild the codestream and extract
the
> parts to decode by reading the main header?. 

Main header contains the progression order information, but it doesn't
include the address information (start position) of each priority level.

Priority field helps the receiver to select the codestream to put into
decoder easily. Without priority field, the receives needs more memory
to rebuild it and has to parse the codestream.

Best Regards,

Futemma

> Cheers,
> Ali 
> -----Original Message-----
> From: Colin Perkins [mailto:[email-masked]] 
> Sent: lundi 14 mai 2007 13:59
> To: IETF AVT WG
> Cc: Roni Even; Tom-PT Taylor
> Subject: [AVT] Working group last call: JPEG-2000 payload format
> 
> This is to announce a working group last call on the RTP Payload  
> Format for JPEG-2000:
> 
> http://www.ietf.org/internet-drafts/draft-ietf-avt-rtp-jpeg2000-15.txt
> http://www.ietf.org/internet-drafts/draft-ietf-avt-rtp-jpeg2000- 
> beam-06.txt
> 
> Please send any final comments on these drafts to the <[email-masked]>  
> mailing list by 28 May 2007. If no substantive comments are received  
> by that time, we will ask the IESG to consider these drafts for  
> publication as Proposed Standard RFCs.
> 
> Note that there is declared IPR on the "beam" draft:
> https://datatracker.ietf.org/public/ipr_detail_show.cgi?ipr_id=697
> 

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: Thomas Wiegand <[email-masked]>
Date: Mon, 9 Jul 2007 11:16:49 +0200

Hello Ali,

Wrt your 1. question, please note that all NAL units within one  
access unit in SVC share the same relative time stamp (in progressive  
pictures) as the pictures they are representing are assumed to be  
sampled/displayed at the same time.

Wrt your 2. question, there are no fragments in SVC like in JPEG2K.  
All scalability in SVC is achieved by dropping complete NAL units (==  
packets).

Best Regards,
Thomas

---
Dr.-Ing. Thomas Wiegand
Head, Image Communication Group
Image Processing Department
Fraunhofer Inst. for Telecommunications - HHI
Einsteinufer 37, D-10587 Berlin, Germany
Phone: +49 30 31002 617,
Mobile: +49 172 381 3814
http: iphome.hhi.de/wiegand
---

---

From: Satoshi Futemma <[email-masked]>
Date: Tue, 10 Jul 2007 22:41:02 +0900

Dear Ali,

> their own Sequence numbering space?.

In J2K codestream, same clock rate must be used
when one codestream is tranfered in multiple RTP sessions.

About sequence number, there is no no relations between rtp sessions,
so each rtp session follows its own numbering.

For example,
   RTP session 1: 3560 3561 3562 ...
   RTP session 2: 103  104  105 ...
   RTP session 3: 65538 65539 65540 ...

Best Regards,

Futemma