Subject: [MMUSIC] Progressing the rid draft (RTP Payload Format Constraints)

From: Flemming Andreasen <[email-masked]>
Date: Mon, 14 Dec 2015 15:09:38 -0500

Greetings MMUSIC

As you know, we have an external dependency and a fairly aggressive 
timeline on the RTP Payload Format Constraints draft 
(https://datatracker.ietf.org/doc/draft-ietf-mmusic-rid/)

We haven't seen much list traffic on this draft since the Yokohama 
meeting and hence would like to remind people to please take a look at 
and post any comments you may have on the draft. If you have read the 
current version of the draft and don't have any comments on it, we would 
like to hear about that too.

Thanks

-- Flemming (as MMUSIC co-chair)

---

From: Stephan Wenger <[email-masked]>
Date: Mon, 14 Dec 2015 20:52:46 +0000

Hi all,
The comments I made in the mmusic session and sent to the list on 11/1 (with unfortunately an uninformative subject line; message attached for convenience) still stand.
Stephan

On 12/14/15, 12:09, "mmusic on behalf of Flemming Andreasen" <[email-masked] on behalf of [email-masked]> wrote:

>_______________________________________________
>mmusic mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/mmusic

Subject: [MMUSIC] Rid draft: suggestion for change to enable dynamic resolution changes
From: Stephan Wenger <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Mon, 2 Nov 2015 01:48:22 +0000
List-Id: Multiparty Multimedia Session Control Working Group <mmusic.ietf.org>

Hi,
As promised, here is a very rough idea how to express rid attributes:
Combine max_width and max_height to max-samples to allow resolution flexibility within given memory constraints.  (This would approximately correspond to the memory aspect of H.26x levels)
Keep max_fps, but define it for variable frame rate codec as an average over a second or so.  (Matches more closely the semantics of levels in H.26x)
Keep max_br as defined
Keep max_pps, but remove the confusing binding of max_pps with max_fps.   Max_pps controls “burstiness” of the codec (something that lives in the HRD parameters in H.26x).  
For codecs with a highly asymmetric behavior of entropy decoding complexity vs. reconstruction complexity (a decade ago I would have said H.264 w/ CABAC :-), you may want something limiting the number of bits per sample or bits per picture or something.
Stephan

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 14 Dec 2015 16:38:15 -0500

> The comments I made in the mmusic session and sent to the list on 11/1 (with unfortunately an uninformative subject line; message attached for convenience) still stand.

Stephan,

>  1. Combine max_width and max_height to max-samples to allow
> resolution flexibility within given memory constraints. (This would
> approximately correspond to the memory aspect of H.26x levels)

By "samples" do you mean "pixels"? If not, how is the mapping from 
pixels to samples to be done?

	Thanks,
	Paul

---

From: Stephan Wenger <[email-masked]>
Date: Mon, 14 Dec 2015 22:03:14 +0000

Yes, by samples I mean pixels, and samples is the more accurate and scientific word for that thing.
S.

---

From: Adam Roach <[email-masked]>
Date: Mon, 14 Dec 2015 16:28:23 -0600

> The comments I made in the mmusic session and sent to the list on 11/1 (with unfortunately an uninformative subject line; message attached for convenience) still stand.

Thanks for pointing them out! I missed them the first time around. 
Responses inline.

>   * Combine max_width and max_height to max-samples to allow
>     resolution flexibility within given memory constraints.  (This
>     would approximately correspond to the memory aspect of H.26x levels)
>

This is the definition of max-fs ("maximum frame size") in the current 
document. Your request basically resolves to a suggestion that we 
eliminate max-width and max-height. Given that support for any given 
parameter is optional, I'm not sure what gain that provides.

With the clarification that you've just described max-fs, do you still 
think a change is called for?

>   * Keep max_fps, but define it for variable frame rate codec as an
>     average over a second or so.  (Matches more closely the semantics
>     of levels in H.26x)
>

To be clear: your proposal is to use "average over a second" rather than 
"inverse of the minimum interval between frames." I don't have a strong 
position on this, and would like to hear from other WG participants on 
the topic.

>   * Keep max_br as defined

I'm good with this suggestion.

>   * Keep max_pps, but remove the confusing binding of max_pps with
>     max_fps.   Max_pps controls “burstiness” of the codec (something
>     that lives in the HRD parameters in H.26x).
>

Could you propose an alternate definition for inclusion in the document? 
The distinction you're trying to make seems pretty subtle, and I'm not 
sure I quite follow.
>
>   * For codecs with a highly asymmetric behavior of entropy decoding
>     complexity vs. reconstruction complexity (a decade ago I would
>     have said H.264 w/ CABAC :-), you may want something limiting the
>     number of bits per sample or bits per picture or something.
>

Given that it *isn't* a decade ago, is it realistic to assume that any 
modern codecs exhibit this kind of strong asymmetry? Keep in mind that 
we're establishing a registry, so this can be added later if one is 
developed in the future.

/a

---

From: Stephan Wenger <[email-masked]>
Date: Mon, 21 Dec 2015 02:23:35 +0000

Hi Adam, all,
Please see inline, and sorry for the late reply.
Stephan

From: Adam Roach <[email-masked]<mailto:[email-masked]>>
Date: Monday, December 14, 2015 at 14:28
To: Stephan Wenger <[email-masked]<mailto:[email-masked]>>, mmusic <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [MMUSIC] Progressing the rid draft (RTP Payload Format Constraints)

Thanks for pointing them out! I missed them the first time around. Responses inline.

  *   Combine max_width and max_height to max-samples to allow resolution flexibility within given memory constraints.  (This would approximately correspond to the memory aspect of H.26x levels)

This is the definition of max-fs ("maximum frame size") in the current document. Your request basically resolves to a suggestion that we eliminate max-width and max-height. Given that support for any given parameter is optional, I'm not sure what gain that provides.

With the clarification that you've just described max-fs, do you still think a change is called for?

<StW>: I still don’t like providing codepoints that invite implementation shortcuts that, in turn, eliminate the option to meaningfully use a video coding tool that has proven its usefulness for decades.  Let me turn the question around: why are these codepoints useful?  Surely you are not going to argue display resolution or window sizes, are you?    </StW>

  *   Keep max_fps, but define it for variable frame rate codec as an average over a second or so.  (Matches more closely the semantics of levels in H.26x)

To be clear: your proposal is to use "average over a second" rather than "inverse of the minimum interval between frames." I don't have a strong position on this, and would like to hear from other WG participants on the topic.

<StW: me too </StW>

  *   Keep max_br as defined

  *   Keep max_pps, but remove the confusing binding of max_pps with max_fps.   Max_pps controls “burstiness” of the codec (something that lives in the HRD parameters in H.26x).

Could you propose an alternate definition for inclusion in the document? The distinction you're trying to make seems pretty subtle, and I'm not sure I quite follow.

<StW: My suggestion is basically editorial.  I find the calculation provided in the bullet confusing, mostly because of the recalculation.  Simply say something like:
“max-pps, for pixel rate in pixels per second calculated over a moving interval of one second. “ (the “calculated” part obviously should be consistent to whatever we come up with regarding max_fps.) </StW>

  *   For codecs with a highly asymmetric behavior of entropy decoding complexity vs. reconstruction complexity (a decade ago I would have said H.264 w/ CABAC :-), you may want something limiting the number of bits per sample or bits per picture or something.

Given that it *isn't* a decade ago, is it realistic to assume that any modern codecs exhibit this kind of strong asymmetry? Keep in mind that we're establishing a registry, so this can be added later if one is developed in the future.

<StW> H.264 is still the dominant codec in this world and will remain so for the foreseeable future.  And, for higher resolutions and relatively low horsepower machine, the question of using CA-VLC versus CABAC can well be do or die.  So while the problem was quite prominent in 2003, it hardly has gone away.  However, historic (or not so historic) precedences aside, you are building a protocol that ought to be future proof.  Taking a known class of problems into account even in the absence of a concrete problem today is good design practice, even taking into account an hopefully clean extension mechanism, no?

The relative decoding complexity of the entropy decoder vis a vis the reconstructor is a problem that has haunted video coding forever.  In its newest incarnation, H.265 “solves" it (in a very crude way) through the introduction of “tier” for a profile/tier/level concept.  This is one area where contributions outside of the traditional video SDOs and their hardware/software vendor politics could actually be truly helpful.

The key observation here is that entropy decoder complexity is largely a function of bitrate, whereas the reconstructor complexity is largely based on the picture size.  This, I believe, is true for all standards codecs since H.261 up to and including all variants discussed in IETF, ITU, MPEG, SMPTE, AOM, google, and whatnot.

So I would suggest to add:

max_bps, maximum bits per sample, float, calculated as an average of all samples of any given coded picture.  Range 0.0001 to 48

0.001 is an arbitrary choice for a quality level that looks impossibly shabby with every video codec currently in existence (when applied to a sufficiently long series of pictures).  48 represents a 3 component sample at 16 bit resolution per component; this is the most advanced operation point used by video codecs running without compression that I’m aware of.   If you don’t like floats, we could use an integer by introducing a suitable divisor.)

</StW>

Stephan

/a