Subject: [AVTCORE] I-D Action: draft-ietf-avtcore-rtp-vvc-07.txt

From: [email-masked]
Date: Tue, 19 Jan 2021 21:05:49 -0800

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Audio/Video Transport Core Maintenance WG of the IETF.

        Title           : RTP Payload Format for Versatile Video Coding (VVC)
        Authors         : Shuai Zhao
                          Stephan Wenger
                          Yago Sanchez
                          Ye-Kui Wang
	Filename        : draft-ietf-avtcore-rtp-vvc-07.txt
	Pages           : 61
	Date            : 2021-01-19

Abstract:
   This memo describes an RTP payload format for the video coding
   standard ITU-T Recommendation H.266 and ISO/IEC International
   Standard 23090-3, both also known as Versatile Video Coding (VVC) and
   developed by the Joint Video Experts Team (JVET).  The RTP payload
   format allows for packetization of one or more Network Abstraction
   Layer (NAL) units in each RTP packet payload as well as fragmentation
   of a NAL unit into multiple RTP packets.  The payload format has wide
   applicability in videoconferencing, Internet video streaming, and
   high-bitrate entertainment-quality video, among other applications.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-avtcore-rtp-vvc/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-avtcore-rtp-vvc-07
https://datatracker.ietf.org/doc/html/draft-ietf-avtcore-rtp-vvc-07

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-avtcore-rtp-vvc-07

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: "shuaiizhao(Shuai Zhao)" <[email-masked]>
Date: Wed, 20 Jan 2021 05:12:43 +0000

Hi Group, Happy 2021.

Version 07 of VVC RTP payload format has the following update:

     *   Removed support of sprop-opi,
        *   Remove editor-note  #5, #6,  #17, updated editor-note 7
     *   Added support of optional parameter: sprop-recv-id
     *   Updated text for recv-ols-id
     *   Removed editor-note #9, #15, #16 based on confirmed value in recent VVC spec
We also attached the previous WG email discussion for those update.

Best,
Shuai Zhao

From: avt <[email-masked]> on behalf of "[email-masked]" <[email-masked]>
Reply-To: "[email-masked]" <[email-masked]>
Date: Tuesday, January 19, 2021 at 21:06
To: "[email-masked]" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: [AVTCORE] I-D Action: draft-ietf-avtcore-rtp-vvc-07.txt(Internet mail)

Abstract:

_______________________________________________
Audio/Video Transport Core Maintenance
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/avt

Subject: [AVTCORE] VVC payload format, open issue cluster #2 (editor-notes #9, #15, and #16)(Internet mail)
From: Stephan Wenger <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Tue, 12 Jan 2021 06:11:54 +0000
List-Id: Audio/Video Transport Core Maintenance <avt.ietf.org>

Hi, 

This is the second in a series of emails in which we authors will try to propose ways to solve open issues indicated in the draft (https://tools.ietf.org/html/draft-ietf-avtcore-rtp-vvc-06). 

Editor notes #9, #15, and #16 each request verification of certain features/numbers derived from the H.266 specification. 

For editor note #9, we suggest replacing the term “level numbers” with “level values”.  Otherwise, the language has been verified as correct by both Yago and Ye-Kui.  Hence we suggest to remove editor note #9.

For editor notes #15 and #16 the numbers are now independently verified by Yago and by Ye-Kui Wang (who is the editor of the relevant part of the H.266 standard).  We indeed have output layer set numbers between  0 and 257, unintuitive as it may appear.  Therefore, we think we should remove the editor notes #15, and #16.

Unless people speak up here, we will make aforementioned changes in the next version.

Thanks,

Stephan

Subject: Re: [AVTCORE] VVC payload format, open issue cluster #1 (SDP for scalability)(Internet mail)
From: Stephan Wenger <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Wed, 6 Jan 2021 06:11:29 +0000
List-Id: Audio/Video Transport Core Maintenance <avt.ietf.org>

All:

As a reminder, please provided comments by the 13th. 

Thanks,

Stephan

From: avt <[email-masked]> on behalf of Stephan Wenger <[email-masked]>
Date: Thursday, December 17, 2020 at 08:31
To: "[email-masked]" <[email-masked]>
Subject: [AVTCORE] VVC payload format, open issue cluster #1 (SDP for scalability)

Hi,

This is the first in a series of emails in which we authors will try to propose ways to solve open issues indicated in the draft (https://tools.ietf.org/html/draft-ietf-avtcore-rtp-vvc-06).  As we have many of those, we will try to organize those open issues into meaningful clusters get piecemeal agreement on each cluster.  Once received, we will update the draft and go for the next cluster.  Sometimes, we will overlap unrelated clusters.

Many of these emails are going to be long, sorry for that.  Many also require quite detailed knowledge of H.266.  Occasionally, we will include some explanatory language on the H.266 feature in question.  If you want to dive deeper, H.266  is now available for free download from https://www.itu.int/itu-t/recommendations/rec.aspx?rec=14336 –have fun reading the 500 pages of super-dense (and small font) text.

The first cluster concerns section 7.1, Media Type Registration, and therein specifically the codepoints and language related to non-temporal scalability support.  This is a hard problem, quite possibly the hardest we need to address in  this exercise.  I apologize in advance for the word count.

As a very brief and oversimplified recap, H.266/VVC supports temporal scalability in all of its profiles except the still image profiles.  In that, H.266 is similar to H.265/HEVC.  However, H.266 also has two scalable profiles (one for  4:2:0 and the another for 4:4:4) that allow

for a powerful spatial/SNR scalability mode.  Many of us believe that these profiles have higher chances of success compared to previous video coding standards, because they are a) not requiring much in terms of decoder implementation complexity  beyond non-scalable implementations, and b) are conceptually similar to what VP9 and AV1 use, and hence familiar to those using modern scalable codecs, like the webrtc folks.  Insofar, we want to support scalability in the payload format from the outset.

If only temporal scalability is supported, the selection of an appropriate sub-layer in the onion-shaped multi-sublayered stream by an SDP-based offer-answer is a one-dimensional problem, and has been solved in places like the HEVC payload  format by an offer-answer exchange involving a single numeric parameter indicative of the highest supported sub-layer.  However, when having all temporal, spatial, and SNR scalability, the optimal selection of an appropriate output layer set (OLS) becomes  a complex multi-dimensional problem, that may have more than one solution for different values of “optimal”, and also requires certain context.  We try to solve this multi-dimensional problem by a combination of a requirement for conditional presence of certain  otherwise optional SDP codepoints when scalability is in use, and the introduction of an additional few code points.  Alternatives we rejected include not supporting scalability, or relying exclusively on the exchange of possibly many variants of parameter  sets, as that could lead to excessively large SDP blobs (easily larger than the 1500 bytes still common as MTU).  Note that the latter option can still be implemented for cases where our restrictions are considered inappropriate.

The design philosophy is as follows:
When using scalability and OLS-based negotiation, the sender must include a Video Parameter Set (VPS) as a sprop-vps or have otherwise pre-arranged knowledge that the VPS is known  by the receiver.  For those unfamiliar with H.266, the VPS includes a “table of content” of the layering structure, and without it, the values of new codepoints (sprop-ols-id and recv-ols-id) could not be interpreted.
sprop-ols-id will typically point to the “best” output layer set—the combination of layers providing the most pleasing reconstructed result—that the sender can support taking into  account all other parameters known at this point of the O/A process.
The value of sprop-ols-id selects one of possibly several OLSs branches in the VPS.  In the reply, using recv-ols-id, the answerer can select that OLS or other OLSs from the same branch  of scalable layers in the VPS to which the sprop-ols-id belongs, but not any other OLSs that may exist in the VPS.  The worst-case fallback will be base-layer only, which is always supported, assuming H.266 and the relevant profiles tiers, and levels are supported.
Sublayers may be negotiated as in the HEVC payload format and in parallel.
The interaction between what’s going on for the (temporal) sub-layers and for the (spatial/SNR) layers is an application logic problem and does not need to be specified, nor documented.  It’s very hard to grok for anyone not intimately familiar with H.266.  
With these design principles, sprop-opi is not needed anymore and will be removed.

Based on this design, the following changes can be made in the draft:

-Remove edt note #5, and remove the mentioning of sprop-opi in the sentence above note 5.

-Remove edt note #6, and remove the mentioning of sprop-opi in the sentence above note 6. 

-sprop-ols-id: add a sentence “if this optional parameter is present, sprop-vps must also be present or its content must be known a priori at the receiver”

-sprop-recv-id: add “sprop-recv-id, when present, must be included only when sprop-ols-id was received and must refer to an output layer set in the VPS that is in the same dependency tree as the OLS referred to by sprop-ols-id.  If this  optional parameter is present, sprop-vps must have been received or its content must be known a priori at the receiver” 

-Remove the sprop-opi language and edt note #17

Please provide comments to this design choice by Jan 6th.  Absent comments, we will implement the changes as described above, also any changes required for consistency, and we will add corresponding language in the currently  empty offer-answer section of the draft.

Thanks,

Stephan