Subject: Re: [clue] How to choose from multiple scenes

From: Paul Kyzivat <[email-masked]>
Date: Thu, 06 Feb 2014 12:08:24 -0500

Any comments on this?

On 1/27/14 6:06 PM, Paul Kyzivat wrote:
> * The problem:
>
> As we discussed in the interim today, it seems unclear what a consumer
> should do when receiving an advertisement with multiple scenes in order
> to get a sufficient representation of the advertised content.
>
> Our simple cases have been where there is one scene for the cameras in
> the room, and another scene for a presentation. In that case, one should
> take something from each scene (typically one CSE from each) to get
> "enough".
>
> That is also true if an MCU acted in "pass through" mode and simply
> included "copies" of all the scenes it receives in advertisements, with
> encodings for them all.
>
> In a "simple" case with an MCU use MCC, it might "pass through" all the
> advertisements it receives, for their spatial info, but not provide any
> encodings for them. Rather, it would provide one or more new scenes
> using MCC to aggregate those sources. Then one would not want to
> configure from the "pass through" scenes, but that isn't an option
> anyway if there are no encodings for them.
>
> Mark has proposed using multiple scenes with MCC to illustrate switched
> cases where some groupings don't have any spatial relationship to one
> another. Some of those examples end up with multiple scenes that do have
> encodings, where it doesn't make sense to configure something from each
> scene.
>
> Today we discussed using priority to solve this, but this isn't really a
> priority problem, it is a problem of *alternatives* that may be equally
> desirable, depending on the resources or desires of the consumer.
>
> IMO the problem is that our CSE mechanism is a way for the advertiser to
> suggest alternatives, but this only works on a per-scene basis. The
> advertiser has no comparable mechanism to suggest alternatives from
> different scenes.
>
> * My proposed solution:
>
> I want to restate a proposal I made a long time ago:
>
> Instead of one CSE list per scene, have one CSE list per-advertisement,
> referencing captures from any scene.
>
> This makes it possible for the advertiser to recommend combinations that
> cross scene boundaries.
>
> * An alternate solution:
>
> Another possibility would be to leave CSEs as they are, but add
> something analogous to a CSE list, but that lists recommended
> combinations of scenes.
>
>      Thanks,
>      Paul
> _______________________________________________
> clue mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/clue
>

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 06 Feb 2014 12:34:22 -0500

I just realized I had missed this reply. See inline.

On 1/27/14 10:33 PM, Christian Groves wrote:
> Hello Paul,
>
> Please see below.
>
> Regards, Christian
>
> On 28/01/2014 10:06 AM, Paul Kyzivat wrote:
>> * The problem:
>>
>> should do when receiving an advertisement with multiple scenes in
>> order to get a sufficient representation of the advertised content.
>>
>> the room, and another scene for a presentation. In that case, one
>> should take something from each scene (typically one CSE from each) to
>> get "enough".
>>
>> included "copies" of all the scenes it receives in advertisements,
>> with encodings for them all.
>>
>> In a "simple" case with an MCU use MCC, it might "pass through" all
>> the advertisements it receives, for their spatial info, but not
>> provide any encodings for them. Rather, it would provide one or more
>> new scenes using MCC to aggregate those sources. Then one would not
>> want to configure from the "pass through" scenes, but that isn't an
>> option anyway if there are no encodings for them.
> [CNG] The idea of the pass through information was that the consumer
> could use any capture attribute (not just spatial ones) to determine
> what media it wanted.

Clearly if there are no encodings, then the captures are there only for 
information and can't be configured.

That is why I called this the "simple" case.

>> Mark has proposed using multiple scenes with MCC to illustrate
>> switched cases where some groupings don't have any spatial
>> relationship to one another. Some of those examples end up with
>> multiple scenes that do have encodings, where it doesn't make sense to
>> configure something from each scene.
> [CNG] I don't think there's anything in the framework that says a
> consumer must choose a capture from a scene.

Of course there is no MUST. The consumer isn't required take *anything*.

The question is how the consumer knows what is required to have a 
"complete" representation of what is in the advertisement. It may not 
want all that, but it is hard to make a good choice without knowing.

That is the point of CSEs - for the advertiser to indicate what it 
considers to be good alternative choices. But CSEs only work within a 
scene. Once there are multiple scenes, the consumer has less guidance on 
what would be good (or sufficient) choices.

The consumer has *some* information:
- if none of the captures in a scene have encodings, then there is
   no need (or way) to select them directly

- if MCC M in scene X references capture C in scene Y, then it
   would be redundant to configure both M and C

ISTM that one should start out assuming that you need all captures (that 
have encodings) in *some* CSE from *each* scene. You can then prune that 
based on the logic above. But is that *enough* to make good decisions? 
At best, the logic to figure that out is complex.

>> Today we discussed using priority to solve this, but this isn't really
>> a priority problem, it is a problem of *alternatives* that may be
>> equally desirable, depending on the resources or desires of the consumer.
> [CNG] I agree I don't think priority is the right solution for this.
>>
>> IMO the problem is that our CSE mechanism is a way for the advertiser
>> to suggest alternatives, but this only works on a per-scene basis. The
>> Instead of one CSE list per scene, have one CSE list
>> per-advertisement, referencing captures from any scene.
>>
>> This makes it possible for the advertiser to recommend combinations
>> that cross scene boundaries.
> [CNG] Perhaps you could give some examples? Does the list need to be
> exhaustive?

Does the CSE list in a scene have to be exhaustive?

I think each entry should be "sufficient" for somebody that can't handle 
a bigger one. The definition of "sufficient" is subjective.

> e.g. the simple case today
> Scene1 CSE1(VC1,VC2,VC3)
>         CSE2(VC4)
> Scene2 CSE3(VC5,presentation1)
>
> Does it mean now the Advertisement would now have to advertise?
> CSE1(VC1,VC2,VC3,VC5)
> CSE2(VC4,VC5)
> CSE3(VC1,VC2,VC3)
> CSE4(VC4)
> CSE5(VC5)

As I said, the definition of "sufficient" is subjective.
IMO the advertisement ought to include at least CSE1,CSE2.

Rather than include the others, I think I would adjust the priorities of 
the individual captures based on my judgement of which I think should be 
kept if they all can't be.

	Thanks,
	Paul

---

From: Christian Groves <[email-masked]>
Date: Fri, 07 Feb 2014 12:24:10 +1100

Hello Paul,

Please see below.

Regards, Christian

On 7/02/2014 4:34 AM, Paul Kyzivat wrote:
> Clearly if there are no encodings, then the captures are there only 
> for information and can't be configured.
>
> scene. Once there are multiple scenes, the consumer has less guidance 
> on what would be good (or sufficient) choices.
>
> ISTM that one should start out assuming that you need all captures 
> (that have encodings) in *some* CSE from *each* scene. You can then 
> prune that based on the logic above. But is that *enough* to make good 
> decisions? At best, the logic to figure that out is complex.
[CNG] The above is probably a valid assumption. I think ultimately a 
Consumer chooses the scene because of the content.
> I think each entry should be "sufficient" for somebody that can't 
> handle a bigger one. The definition of "sufficient" is subjective.
[CNG] A Scene Level CSE only specifies the alternate representations for 
that scene. No sufficient or exhaustive just what it can support.

>> Scene2 CSE3(VC5,presentation1)
[CNG] Is in fact exhaustive because it basically implies the 
combinations I listed below.
> IMO the advertisement ought to include at least CSE1,CSE2.
[CNG] My point wasn't about "sufficient" it was about matching the 
expressive power that we have today. The combinatorial issue becomes 
worse with the more Scenes that you have.

If we go down this track it may be better to indicate which scenes go 
together rather than CSEs.
>
> Rather than include the others, I think I would adjust the priorities 
> of the individual captures based on my judgement of which I think 
> should be kept if they all can't be.
>
>     Thanks,
>     Paul
>
>>> * An alternate solution:
>>>

---

From: "Duckworth, Mark" <[email-masked]>
Date: Fri, 7 Feb 2014 12:26:22 -0800

Hi Paul,

You describe the issue very well, but I don't agree with a couple things.  You wrote:
> have encodings) in *some* CSE from *each* scene.
I don't think that is an appropriate thing to assume.  The case I was thinking of, and we have been talking about at design team meetings, is like this:
Scene 1 - current active speaker
Scene 2 - previous active speaker
Scene 3 - previous previous active speaker
and so on...
Where each scene could have multiple switched MCCs representing the scene, and the provider has some flexibility to group together different individual captures to include in the MCCs of a scene, depending on synchID and who is talking.
In this case, because of the policy attributes describing the MCCs, I think it makes perfect sense for a consumer to ask for all the video MCCs from Scene 1, and none of them from Scene 2 or Scene 3.  The consumer has enough information to know this is a good choice if it wants to show video from the current active speaker, including multiple video captures with spatial relationship when the current speaker is in a multi-camera room.  So in this case the consumer already has enough information to make a choice.

I agree there could be other situations where the MCC policy attributes aren't enough information for the consumer to make a good choice.  This is where I thought the media capture priority attribute should be used.  I thought this was the purpose of adding the priority attribute in the first place.  From the framework:

7.1.1.12. Priority
The priority attribute indicates a relative priority between different Media Captures.  The Provider sets this priority, and the Consumer MAY use the priority to help decide which captures it wishes to receive.
The "priority" attribute is an integer which indicates a relative priority between Captures. For example it is possible to assign a priority between two presentation Captures that would allow a remote endpoint to determine which presentation is more important. Priority is assigned at the individual capture level. It represents the Provider's view of the relative priority between Captures with a priority. 

I don't understand why you say this isn't a priority issue, but rather an alternative issue.  It seems to me your proposal of advertising different combinations of captures or CSEs across scene boundaries is just another way for the provider to express priorities.

Mark

> -----Original Message-----
> From: clue [mailto:[email-masked]] On Behalf Of Paul Kyzivat
> Sent: Thursday, February 06, 2014 12:34 PM
> To: [email-masked]
> Subject: Re: [clue] How to choose from multiple scenes
> 
> >> As we discussed in the interim today, it seems unclear what a
> >> consumer should do when receiving an advertisement with multiple
> >> scenes in order to get a sufficient representation of the advertised
> content.
> >>
> >> Our simple cases have been where there is one scene for the cameras
> >> in the room, and another scene for a presentation. In that case, one
> >> should take something from each scene (typically one CSE from each)
> >> to get "enough".
> >>
> >> multiple scenes that do have encodings, where it doesn't make sense
> >> to configure something from each scene.
> "complete" representation of what is in the advertisement. It may not want
> all that, but it is hard to make a good choice without knowing.
> 
> That is the point of CSEs - for the advertiser to indicate what it considers to
> be good alternative choices. But CSEs only work within a scene. Once there
> are multiple scenes, the consumer has less guidance on what would be good
> (or sufficient) choices.
> 
> >> Today we discussed using priority to solve this, but this isn't
> >> really a priority problem, it is a problem of *alternatives* that may
> >> be equally desirable, depending on the resources or desires of the
> consumer.
> >> to suggest alternatives, but this only works on a per-scene basis.
> >> The advertiser has no comparable mechanism to suggest alternatives
> >> from different scenes.
> >>
> >> * My proposed solution:
> >>
> I think each entry should be "sufficient" for somebody that can't handle a
> bigger one. The definition of "sufficient" is subjective.
> 
> Rather than include the others, I think I would adjust the priorities of the
> individual captures based on my judgement of which I think should be kept if
> they all can't be.
> 
> 	Thanks,
> 	Paul
> 
> >> * An alternate solution:
> >>

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 07 Feb 2014 16:44:20 -0500

> Scene 3 - previous previous active speaker

I understand your *intent* here. But I don't know how the consumer acts 
on that. It is really part of the point I am trying to make.

How does the consumer *know* the above, and take it into account?

Are the *scenes* tagged to indicate that? (I don't think so.)

> In this case, because of the policy attributes describing the MCCs, I think it makes perfect sense for a consumer to ask for all the video MCCs from Scene 1, and none of them from Scene 2 or Scene…

I'm having difficulty imagining how to construct an algorithm for that.

ISTM in this case that taking a cse from every scene will give the best 
user experience *if* the consumer has enough display resources to show 
all these.

If not, then it gets complex to sort out the tradeoffs.

In this case, considering that a capture with "current active speaker" 
policy is more important than one with "previous active speaker".

So I would suggest that the advertisement mark all the "current active 
speaker" captures with highest priority, and the others with descending 
priorities.

Then, the consumer could start out assuming it should *try* for one CSE 
from each scene. It can look at different combinations, taking one from 
each scene, and evaluate what it can do with that combination. For each one:

If it can't handle all the captures from the selected cses, it can start 
casting out lower priority ones until it can handle it. Then it can 
assign a score, depending on the priorities of those it took, the 
priorities of those it had to abandon, and how much it had to "adjust" 
the ones it took to fit on its displays.

And it can repeat that for all the combinations of one cse from each 
scene. And then pick out the combination with the best score.

This is all very heuristic. Will take a lot of thought and testing to 
come up with something that does well. (It would be interesting to work 
this out in detail for some specific consumer configurations and see if 
it can actually find good renderings.)

Having the advertisement provide a single set of CSEs across all the 
scenes would "help" in that it would reduce the number of combinations 
the consumer had to evaluate.

> I don't understand why you say this isn't a priority issue, but rather an alternative issue.  It seems to me your proposal of advertising different combinations of captures or CSEs across scene bou…

Surely it is a form of prioritization. But it can be more sophisticated 
than simply assigning a priority value to each individual capture.

But consider, if prioritization were sufficient, we wouldn't need to 
have CSEs at all. We could simply have prioritized captures. ISTM that 
the reasons that CSEs are more powerful than priorities within a scene 
should still apply outside a scene.

As I described above, I *think* priorities plus per-scene cse lists can 
solve your use case. But the processing to use it that way is complex.

	Thanks,
	Paul

---

From: "Duckworth, Mark" <[email-masked]>
Date: Fri, 7 Feb 2014 14:58:10 -0800

Hi Paul,
Comments inline.
I'm open to your idea of a single set of CSEs across all scenes, but I'm trying to understand if it will really be a scalable and less complex way to solve these issues.
Do you think it would help if you presented this idea at the design team meeting next week?

Mark

> From: Paul Kyzivat [mailto:[email-masked]]
> Sent: Friday, February 07, 2014 4:44 PM
> To: Duckworth, Mark; [email-masked]
> >> (that have encodings) in *some* CSE from *each* scene.
> > I don't think that is an appropriate thing to assume.  The case I was thinking
> of, and we have been talking about at design team meetings, is like this:
> > Scene 3 - previous previous active speaker and so on...
> 
> I understand your *intent* here. But I don't know how the consumer acts on
> that. It is really part of the point I am trying to make.

 [Duckworth, Mark] From the MCC policy attribute values.
 
> > Where each scene could have multiple switched MCCs representing the
> scene, and the provider has some flexibility to group together different
> individual captures to include in the MCCs of a scene, depending on synchID
> and who is talking.
> > In this case, because of the policy attributes describing the MCCs, I think it
> makes perfect sense for a consumer to ask for all the video MCCs from Scene
> 1, and none of them from Scene 2 or Scene 3.  The consumer has enough
> information to know this is a good choice if it wants to show video from the
> current active speaker, including multiple video captures with spatial
> relationship when the current speaker is in a multi-camera room.  So in this
> case the consumer already has enough information to make a choice.
> 

[Duckworth, Mark] Yeah, it can get complicated.  But is it good enough for equipment from different vendors to interoperate in a reasonable way, even if it isn't the "best" way?  I think so.  I still think my proposed "consumer makes all the switching decisions" approach can solve a lot of these problems.

> ISTM in this case that taking a cse from every scene will give the best user
> experience *if* the consumer has enough display resources to show all
> these.

[Duckworth, Mark] I think "best experience" is subjective here.  I think the consumer has enough information to make a good choice for a variety of different consumer capability levels in terms of number of decoders and displays it has.  Suppose the consumer can receive no more than 3 video encodings.  It could ask for the 3 MCCs from Scene 1, or it could ask for 1 MCC from each scene.  But I would think it would do that only if each scene had a CSE with one MCC in it.  Each choice could give the "best" experience, depending on what the consumer is trying to present to the human users.

> speaker" captures with highest priority, and the others with descending

[Duckworth, Mark] that's what I was thinking, but even that shouldn't be necessary because the consumer can make its own priority decision based on the policy attribute.

> from each scene. It can look at different combinations, taking one from each
> scene, and evaluate what it can do with that combination. For each one:
> 
> If it can't handle all the captures from the selected cses, it can start casting
> out lower priority ones until it can handle it. Then it can assign a score,
> depending on the priorities of those it took, the priorities of those it had to
> abandon, and how much it had to "adjust"
> And it can repeat that for all the combinations of one cse from each scene.
> And then pick out the combination with the best score.
> 
> This is all very heuristic. Will take a lot of thought and testing to come up with
> something that does well. (It would be interesting to work this out in detail
> for some specific consumer configurations and see if it can actually find good
> renderings.)
> 
> Having the advertisement provide a single set of CSEs across all the scenes
> would "help" in that it would reduce the number of combinations the
> consumer had to evaluate.

[Duckworth, Mark] I'm not sure it would reduce anything.  As Christian said, "The combinatorial issue becomes worse with the more Scenes that you have".  I'm concerned that trying to add another layer of alternatives in the advertisement, that crosses scene boundaries, won't scale well.  Or do you think it is really simpler than that?  Are you envisioning the single set of CSEs could be kept small, even with many scenes?

> > I agree there could be other situations where the MCC policy attributes
> aren't enough information for the consumer to make a good choice.  This is
> where I thought the media capture priority attribute should be used.  I
> thought this was the purpose of adding the priority attribute in the first
> place.  From the framework:
> >
> > 7.1.1.12. Priority
> > The priority attribute indicates a relative priority between different Media
> Captures.  The Provider sets this priority, and the Consumer MAY use the
> priority to help decide which captures it wishes to receive.
> > The "priority" attribute is an integer which indicates a relative priority
> between Captures. For example it is possible to assign a priority between
> two presentation Captures that would allow a remote endpoint to determine
> which presentation is more important. Priority is assigned at the individual
> capture level. It represents the Provider's view of the relative priority
> between Captures with a priority.
> >
> > I don't understand why you say this isn't a priority issue, but rather an
> alternative issue.  It seems to me your proposal of advertising different
> combinations of captures or CSEs across scene boundaries is just another
> way for the provider to express priorities.
> 
> Surely it is a form of prioritization. But it can be more sophisticated than
> simply assigning a priority value to each individual capture.
> 
> But consider, if prioritization were sufficient, we wouldn't need to have CSEs
> at all. We could simply have prioritized captures. ISTM that the reasons that
> CSEs are more powerful than priorities within a scene should still apply
> outside a scene.

[Duckworth, Mark] That makes sense too, at a high level, but I'm having trouble understanding how to put it in practice.
 
> As I described above, I *think* priorities plus per-scene cse lists can solve
> your use case. But the processing to use it that way is complex.

[Duckworth, Mark] So are you saying your idea of using a single set of CSEs across all scenes can solve the same problems in a less complex way?  I'm open to that, but still not sure it would be any less complex.
 
> >>>> in the room, and another scene for a presentation. In that case,
> >>>> one should take something from each scene (typically one CSE from
> >>>> each) to get "enough".
> >>>>
> >>>> provide any encodings for them. Rather, it would provide one or
> >>>> more new scenes using MCC to aggregate those sources. Then one
> >>>> would not want to configure from the "pass through" scenes, but
> >>>> that isn't an option anyway if there are no encodings for them.
> decisions?
> >>>> really a priority problem, it is a problem of *alternatives* that
> >>>> may be equally desirable, depending on the resources or desires of
> >>>> the
> >> consumer.
> >>>> IMO the problem is that our CSE mechanism is a way for the
> >>>> advertiser to suggest alternatives, but this only works on a per-scene
> basis.

---

From: Paul Kyzivat <[email-masked]>
Date: Sat, 08 Feb 2014 13:19:54 -0500

> I'm open to your idea of a single set of CSEs across all scenes, but I'm trying to understand if it will really be a scalable and less complex way to solve these issues.

Me too. I'm not certain that it will, but my gut says so.
It just seems to me that the idea of enumerating useful combinations of 
captures in the advertisement is conceptually unrelated to what scene 
they are in, and that for the consumer the process then by default 
devolves to just choosing the one best cse that works for it, rather 
than having to choose from among combinations in different scenes.

> Do you think it would help if you presented this idea at the design team meeting next week?

I'm not convinced that call time is very helpful for this.
To be useful I would probably need to create several use cases, and show 
them both ways, and then for people to try to shoot them down.
I think that kind of thing is better done by email. (And I don't have 
time to put together sufficient use cases before Tuesday.)

More below.

> [Duckworth, Mark] Yeah, it can get complicated.  But is it good enough for equipment from different vendors to interoperate in a reasonable way, even if it isn't the "best" way?  I think so.  I sti…

I know Cisco used to have a goal to make its equipment work "better 
together". But in a sense that is exactly the thing that standards are 
working against.

With CLUE, *how* would a vendor's equipment do better with other of its 
own, and just be "good enough" with others? Ultimately you can only do 
as good as the provided information allows you to do. I can see a few 
things:

- a vendor is likely to make its equipment in configurations that
   are designed to be easily compatible with one another. E.g., just
   a small set of configurations, like 1,2, or 3 screens, all same
   size, all side by side with consistent spacing.

- a vendor could use a consistent style to structure its advertisements.
   E.g., Always have one scene for room cameras and one more for
   presentations.

- a vendor could build an algorithm for constructing configurations
   that special cases advertisements structured the way it does them.
   Then it could have very crude algorithms for anything else, or
   just fail for anything else.

- a vendor could build an algorithm for constructing configurations
   that makes "leaps of faith" that aren't justified solely by what
   is in an advertisement, about how the captures in the advertisement
   work. (E.g., about the algorithm for switching, or how things are
   laid out in a composition.)

- a vendor could add proprietary information to advertisements, or to
   the sip signaling, to convey extra information that allows one of
   its own consuming endpoints to do a better job of constructing a
   config than would be possible with just the standard information.

Odds are that we will see all of the above. Certainly there are inherent 
advantages when the advertisement happens to contain an alternative that 
exactly matches the equipment in the consuming end. And that is more 
likely to happen with equipment from one vendor.

But beyond that, IMO, if we do our job well, then it should be possible 
to construct a general purpose algorithm that will do as well as 
algorithms that are special cased or that make "leaps of faith" or use 
proprietary information.

> [Duckworth, Mark] I think "best experience" is subjective here.  I think the consumer has enough information to make a good choice for a variety of different consumer capability levels in terms of…

I agree that "best experience" is subjective. But I suspect if we worked 
out a lot of use cases, we could come close to agreeing to what the 
"best experience" would be, in the absence of explicit input from the 
end users in the room.

I think what you are talking about is that there are a *set* of 
"plausible" configurations, that it might make sense to offer as a menu 
to the end user.

> [Duckworth, Mark] that's what I was thinking, but even that shouldn't be necessary because the consumer can make its own priority decision based on the policy attribute.

Certainly it *can*. But then it must make a tradeoff between the 
importance of the policy and the importance of the priority in the 
decision. Lacking any understanding of the motivation for the priority 
assignments, it is hard to trade those off against one another.

> [Duckworth, Mark] I'm not sure it would reduce anything.  As Christian said, "The combinatorial issue becomes worse with the more Scenes that you have".  I'm concerned that trying to add another la…

With separate CSE lists in each scene, the *consumer* has to evaluate 
all the combinations of one CSE from each scene.

With a single CSE list, the *advertiser* has to consider all of those 
when constructing the advertisement. But it doesn't have to *include* 
them all in the advertisement. My gut tells me that when there are only 
a few then perhaps they are all interesting, but when they are many, 
that only a few will be interesting enough to include. The advertiser is 
pruning the choices, using information that it has that isn't 
necessarily available to the consumer.

If the advertiser is typically going to include all the combinations, 
then going this way is a bad choice. And if the advertiser is going to 
limit what it includes because it is too much work, or makes the 
advertisement too big, even though it thinks the ones it is omitting 
would be useful, then this is also a bad idea.

> [Duckworth, Mark] So are you saying your idea of using a single set of CSEs across all scenes can solve the same problems in a less complex way?  I'm open to that, but still not sure it would be an…

I think it is less complex for the consumer to evaluate a single list 
than to evaluate combinations of choices from multiple lists. 
(Enumerating all the combinations and generating an equivalent single 
list to evaluate is of course a straightforward programming job. But I 
wonder if some implementers might instead go for "special casing".)

When constructing a single list, rather than several, and choosing which 
combinations to include, the advertiser can use knowledge that can't 
otherwise be described in the advertisement and used by the consumer to 
choose.

	Thanks,
	Paul

---

From: Christian Groves <[email-masked]>
Date: Mon, 10 Feb 2014 11:56:01 +1100

Hello Paul,

I think some uses cases would be good.

I'd like to see a third case where we'd have something like a "Scene Set 
Entry" (SSE). This would indicate what scenes would make up a "full' 
description of an endpoint.

i.e. Scene1 (CSE1)
      Scene2 (CSE2)
      Scene3 (CSE3)
      SSE [(Scene1,Scene2),(Scene1)]
A consumer would know it should pick CSE1 and CSE2 for a complete 
description or CSE3.

I think this is along the lines of your single CSE.

Regards, Christian

On 9/02/2014 5:19 AM, Paul Kyzivat wrote:
>> I'm open to your idea of a single set of CSEs across all scenes, but 
>> I'm trying to understand if it will really be a scalable and less 
>> complex way to solve these issues.
>
> It just seems to me that the idea of enumerating useful combinations 
> of captures in the advertisement is conceptually unrelated to what 
> scene they are in, and that for the consumer the process then by 
> default devolves to just choosing the one best cse that works for it, 
> rather than having to choose from among combinations in different scenes.
>
> To be useful I would probably need to create several use cases, and 
> show them both ways, and then for people to try to shoot them down.
>>>> You describe the issue very well, but I don't agree with a couple 
>>>> things.  You
>>> wrote:
>> [Duckworth, Mark] Yeah, it can get complicated.  But is it good 
>> enough for equipment from different vendors to interoperate in a 
>> reasonable way, even if it isn't the "best" way?  I think so.  I 
>> still think my proposed "consumer makes all the switching decisions" 
>> approach can solve a lot of these problems.
>
> With CLUE, *how* would a vendor's equipment do better with other of 
> its own, and just be "good enough" with others? Ultimately you can 
> only do as good as the provided information allows you to do. I can 
> see a few things:
>
> Odds are that we will see all of the above. Certainly there are 
> inherent advantages when the advertisement happens to contain an 
> alternative that exactly matches the equipment in the consuming end. 
> And that is more likely to happen with equipment from one vendor.
>
> But beyond that, IMO, if we do our job well, then it should be 
> possible to construct a general purpose algorithm that will do as well 
> as algorithms that are special cased or that make "leaps of faith" or 
> use proprietary information.
>
>> [Duckworth, Mark] I think "best experience" is subjective here. I 
>> think the consumer has enough information to make a good choice for a 
>> variety of different consumer capability levels in terms of number of 
>> decoders and displays it has.  Suppose the consumer can receive no 
>> more than 3 video encodings.  It could ask for the 3 MCCs from Scene 
>> 1, or it could ask for 1 MCC from each scene.  But I would think it 
>> would do that only if each scene had a CSE with one MCC in it.  Each 
>> choice could give the "best" experience, depending on what the 
>> consumer is trying to present to the human users.
>
> I agree that "best experience" is subjective. But I suspect if we 
> worked out a lot of use cases, we could come close to agreeing to what 
> the "best experience" would be, in the absence of explicit input from 
> the end users in the room.
>
> "plausible" configurations, that it might make sense to offer as a 
> menu to the end user.
>
>> [Duckworth, Mark] that's what I was thinking, but even that shouldn't 
>> be necessary because the consumer can make its own priority decision 
>> based on the policy attribute.
>
>>> scenes
>> [Duckworth, Mark] I'm not sure it would reduce anything.  As 
>> Christian said, "The combinatorial issue becomes worse with the more 
>> Scenes that you have".  I'm concerned that trying to add another 
>> layer of alternatives in the advertisement, that crosses scene 
>> boundaries, won't scale well.  Or do you think it is really simpler 
>> than that?  Are you envisioning the single set of CSEs could be kept 
>> small, even with many scenes?
>
> them all in the advertisement. My gut tells me that when there are 
> only a few then perhaps they are all interesting, but when they are 
> many, that only a few will be interesting enough to include. The 
> advertiser is pruning the choices, using information that it has that 
> isn't necessarily available to the consumer.
>
>>> have CSEs
>> [Duckworth, Mark] That makes sense too, at a high level, but I'm 
>> having trouble understanding how to put it in practice.
>>
>> [Duckworth, Mark] So are you saying your idea of using a single set 
>> of CSEs across all scenes can solve the same problems in a less 
>> complex way?  I'm open to that, but still not sure it would be any 
>> less complex.
>
> When constructing a single list, rather than several, and choosing 
> which combinations to include, the advertiser can use knowledge that 
> can't otherwise be described in the advertisement and used by the 
> consumer to choose.
>
>     Thanks,
>     Paul
>
>>>     Thanks,
>>>     Paul
>>>
>>>> Mark
>>>>

---

From: "Duckworth, Mark" <[email-masked]>
Date: Mon, 10 Feb 2014 07:49:00 -0800

Hello Christian,

I don't understand your proposal for "Scene Set Entry".  Can you try explaining again?
>       SSE [(Scene1,Scene2),(Scene1)]
What does this mean?

Mark

> From: clue [mailto:[email-masked]] On Behalf Of Christian Groves
> Sent: Sunday, February 09, 2014 7:56 PM
> To: [email-masked]
> >>> I understand your *intent* here. But I don't know how the consumer
> >>> acts on that. It is really part of the point I am trying to make.
> >>> scene, and the provider has some flexibility to group together
> >>> different individual captures to include in the MCCs of a scene,
> >>> depending on synchID and who is talking.
> >>> makes perfect sense for a consumer to ask for all the video MCCs
> >>> from Scene 1, and none of them from Scene 2 or Scene 3.  The
> >>> consumer has enough information to know this is a good choice if it
> >>> wants to show video from the current active speaker, including
> >>> multiple video captures with spatial relationship when the current
> >>> speaker is in a multi-camera room.  So in this case the consumer
> >>> already has enough information to make a choice.
> >>>
> >>> ISTM in this case that taking a cse from every scene will give the
> >>> best user experience *if* the consumer has enough display resources
> >>> to show all these.
> >>
> >>> So I would suggest that the advertisement mark all the "current
> >>> active speaker" captures with highest priority, and the others with
> >>> descending priorities.
> >>
> >>> Then, the consumer could start out assuming it should *try* for one
> >>> CSE from each scene. It can look at different combinations, taking
> >>> one from each scene, and evaluate what it can do with that
> >>> combination. For each one:
> >>>
> >>> If it can't handle all the captures from the selected cses, it can
> >>> start casting out lower priority ones until it can handle it. Then
> >>> it can assign a score, depending on the priorities of those it took,
> >>> the priorities of those it had to abandon, and how much it had to
> >>> "adjust"
> >>> This is all very heuristic. Will take a lot of thought and testing
> >>> to come up with something that does well. (It would be interesting
> >>> to work this out in detail for some specific consumer configurations
> >>> and see if it can actually find good
> >>> renderings.)
> >>>
> >>> scenes would "help" in that it would reduce the number of
> >>> combinations the consumer had to evaluate.
> >>
> >>> where I thought the media capture priority attribute should be used.
> >>> I thought this was the purpose of adding the priority attribute in
> >>> the first place.  From the framework:
> >>>>
> >>>> 7.1.1.12. Priority
> >>> Captures.  The Provider sets this priority, and the Consumer MAY use
> >>> the priority to help decide which captures it wishes to receive.
> >>> between Captures. For example it is possible to assign a priority
> >>> between two presentation Captures that would allow a remote
> endpoint
> >>> to determine which presentation is more important. Priority is
> >>> assigned at the individual capture level. It represents the
> >>> Provider's view of the relative priority between Captures with a
> >>> priority.
> >>>>
> >>> alternative issue.  It seems to me your proposal of advertising
> >>> different combinations of captures or CSEs across scene boundaries
> >>> is just another way for the provider to express priorities.
> >>>
> >>> Surely it is a form of prioritization. But it can be more
> >>> sophisticated than simply assigning a priority value to each
> >>> individual capture.
> >>>
> >>> have CSEs at all. We could simply have prioritized captures. ISTM
> >>> that the reasons that CSEs are more powerful than priorities within
> >>> a scene should still apply outside a scene.
> >>
> >>> As I described above, I *think* priorities plus per-scene cse lists
> >>> can solve your use case. But the processing to use it that way is
> >>> complex.
> >>
> >>>>>>> Our simple cases have been where there is one scene for the
> >>>>>>> cameras in the room, and another scene for a presentation. In
> >>>>>>> that case, one should take something from each scene (typically
> >>>>>>> one CSE from
> >>>>>>> each) to get "enough".
> >>>>>>>
> >>>>>>> That is also true if an MCU acted in "pass through" mode and
> >>>>>>> simply included "copies" of all the scenes it receives in
> >>>>>>> advertisements, with encodings for them all.
> >>>>>>>
> >>>>>>> In a "simple" case with an MCU use MCC, it might "pass through"
> >>>>>>> all the advertisements it receives, for their spatial info, but
> >>>>>>> not provide any encodings for them. Rather, it would provide one
> >>>>>>> or more new scenes using MCC to aggregate those sources. Then
> >>>>>>> one would not want to configure from the "pass through" scenes,
> >>>>>>> but that isn't an option anyway if there are no encodings for them.
> >>>>>> [CNG] The idea of the pass through information was that the
> >>>>>> consumer could use any capture attribute (not just spatial ones)
> >>>>>> to determine what media it wanted.
> >>>>>
> >>>>> Clearly if there are no encodings, then the captures are there
> >>>>> only for information and can't be configured.
> >>>>>
> >>>>>>> multiple scenes that do have encodings, where it doesn't make
> >>>>>>> sense to configure something from each scene.
> >>>>> "complete" representation of what is in the advertisement. It may
> >>>>> not want all that, but it is hard to make a good choice without
> knowing.
> >>>>>
> >>>>> considers to be good alternative choices. But CSEs only work
> >>>>> within a scene. Once there are multiple scenes, the consumer has
> >>>>> less guidance on what would be good (or sufficient) choices.
> >>>>>
> >>>>> (that have encodings) in *some* CSE from *each* scene. You can
> >>>>> then prune that based on the logic above. But is that *enough* to
> >>>>> make good
> >>> decisions?
> >>>>>>> really a priority problem, it is a problem of *alternatives*
> >>>>>>> that may be equally desirable, depending on the resources or
> >>>>>>> desires of the
> >>>>> consumer.
> >>>>>>> The advertiser has no comparable mechanism to suggest
> >>>>>>> alternatives from different scenes.
> >>>>>>>
> >>>>>>> * My proposed solution:
> >>>>>>>
> >>>>>>> This makes it possible for the advertiser to recommend
> >>>>>>> combinations that cross scene boundaries.
> >>>>>> [CNG] Perhaps you could give some examples? Does the list need to
> >>>>>> be exhaustive?
> >>>>>
> >>>>> Rather than include the others, I think I would adjust the
> >>>>> priorities of the individual captures based on my judgement of
> >>>>> which I think should be kept if they all can't be.
> >>>>>
> >>>>>     Thanks,
> >>>>>     Paul
> >>>>>
> >>>>>>> * An alternate solution:
> >>>>>>>

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 10 Feb 2014 11:45:13 -0500

> I think this is along the lines of your single CSE.

IIUC, that is a way to distinguish which scenes need to be considered. 
Mentioned that as a possibility in passing in one of my messages.

It still leaves the consumer with a combinatorial analysis.

Another possibility is:

- Leave existing per-scene CSE lists as-is.

- Add another advertisement-wide CSE list.
   The entries in it could use shorthand, by referencing CSEs from
   the individual scenes.

E.g.,

	Scene1 (CSE11, CSE12, CSE13)
	Scene2 (CSE21, CSE22)
	Scene3 (CSE31)
	Scene4 (CSE41)

	Global-CSE-List (
	  CSEg1(CSE11, CSE21, CSE31)
	  CSEg2(CSE12, CSE22, CSE31)
	  CSEg3(CSE13, CSE22, CSE31)
	)

Here the global CSE list has recommended a subset of the possible 
combinations of CSEs from the different scenes, and has omitted Scene4 
altogether. This is a value judgement of what combinations make sense.

Note that I would still allow the CSEs in the global list to reference 
individual captures if desired. But referencing other CSEs is a 
convenient shorthand.

	Thanks,
	Paul

>>>> acts on
>>>> different
>>>> from Scene
>>>> best user
>>>> start casting
>>>> to come up with
>>>> between
>>>> different
>>>> sophisticated than
>>>> can solve

---

From: "Duckworth, Mark" <[email-masked]>
Date: Mon, 10 Feb 2014 09:23:07 -0800

Hi Paul,
Your proposal here is what I was thinking of when you first suggested something like this last week.  I think this could be useful, even in advertisements with many scenes, without a scalability problem.  I'd like to discuss this more, after the design team meeting this week.
Mark

> Sent: Monday, February 10, 2014 11:45 AM
> To: [email-masked]

... snip ...

> Note that I would still allow the CSEs in the global list to reference individual
> captures if desired. But referencing other CSEs is a convenient shorthand.
> 
> 	Thanks,
> 	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 10 Feb 2014 12:54:25 -0500

> Your proposal here is what I was thinking of when you first suggested something like this last week.  I think this could be useful, even in advertisements with many scenes, without a scalability pr…

Sure.

Note there are variations on how this could be used:

- As I showed, with CSEs in the scenes, referenced from
   a global CSE list

- No CSEs in scenes. The global CSE list directly references captures
   in the various scenes

- No global CSEs, just CSEs in scenes (what have had till now)

- No CSEs anywhere, just captures in scenes. The consumer is on
   his own.

We could allow as few or as many of those as we wish.

	Thanks,
	Paul

---

From: Christian Groves <[email-masked]>
Date: Tue, 11 Feb 2014 21:02:23 +1100

Hello Paul and Mark,

Please see my comments below.

Regards, Christian

> Mentioned that as a possibility in passing in one of my messages.
[CNG] I thought that knowing which scenes go together was the core of 
the issue. An Advertiser says here's the set of scenes that go together. 
The Advertiser uses the CSEs to indicate the different rendering options 
for each scene.
>       CSEg3(CSE13, CSE22, CSE31)
[CNG] The issue is that currently we do not assign any sort of ID to 
CSEs. There hasn't been a reason to as the consumer selects the captures 
it wants.

I think the above complicates things. With this we are now saying that 
which scenes are chosen depend on the rendering of each of the scenes. 
Our current assumption is that a CSE represents the "whole scene". Each 
CSE is effectively a different rendering of this whole. With the above 
we are now changing that to imply that a CSE may be a part of a scene 
because which other Scenes/CSEs you chose is based on that rendering. 
Its not a syntax issue only we're playing with the semantics of what we 
have also.

>>>>> active
>>>>>>>>> cameras
>>>>>>>>> simply
>>>>>>>> consumer
>>>>>>> within a
>>>>>>> priorities

---

From: Christian Groves <[email-masked]>
Date: Wed, 12 Feb 2014 09:36:38 +1100

I stand corrected on my point re: CSE not having IDs.

However given that we have the STS which may list CSEs that can be 
provided together do we another thing?

Christian

On 11/02/2014 9:02 PM, Christian Groves wrote:
>> IIUC, that is a way to distinguish which scenes need to be 
>> considered. Mentioned that as a possibility in passing in one of my 
>> messages.
> the issue. An Advertiser says here's the set of scenes that go 
> together. The Advertiser uses the CSEs to indicate the different 
> rendering options for each scene.
>>
> CSEs. There hasn't been a reason to as the consumer selects the 
> captures it wants.
>
> Our current assumption is that a CSE represents the "whole scene". 
> Each CSE is effectively a different rendering of this whole. With the 
> above we are now changing that to imply that a CSE may be a part of a 
> scene because which other Scenes/CSEs you chose is based on that 
> rendering. Its not a syntax issue only we're playing with the 
> semantics of what we have also.
>
>
>>
>> combinations of CSEs from the different scenes, and has omitted 
>> Scene4 altogether. This is a value judgement of what combinations 
>> make sense.
>>
>> Note that I would still allow the CSEs in the global list to 
>> reference individual captures if desired. But referencing other CSEs 
>> is a convenient shorthand.
>>
>>     Thanks,
>>     Paul
>>
>>> Regards, Christian
>>>
>>>>>>>> then
>>>>>>>>>> that

---

From: "Duckworth, Mark" <[email-masked]>
Date: Wed, 19 Feb 2014 13:36:15 -0800

Responses to Christian's comments below.
Mark

> Sent: Tuesday, February 11, 2014 5:02 AM
> To: [email-masked]
> >> I'd like to see a third case where we'd have something like a "Scene
> >> Set Entry" (SSE). This would indicate what scenes would make up a "full'
> [CNG] I thought that knowing which scenes go together was the core of the
> issue. An Advertiser says here's the set of scenes that go together.

[Duckworth, Mark] That's not what I was thinking.  I thought the issue was how the consumer could choose captures from multiple scenes when it is not capable of choosing the "biggest" CSE from each scene.  I don't think it is about "which scenes go together".

> The Advertiser uses the CSEs to indicate the different rendering options for
> each scene.

[Duckworth, Mark] Yes.

> [CNG] The issue is that currently we do not assign any sort of ID to CSEs.
> There hasn't been a reason to as the consumer selects the captures it wants.
>
> I think the above complicates things. With this we are now saying that which
> scenes are chosen depend on the rendering of each of the scenes.

[Duckworth, Mark] I guess so, because if the consumer is limited in how many captures it can receive, then there is a tradeoff between receiving more scenes with few captures per scene vs. fewer scenes with more captures per scene.  But I think Paul's proposal is simplifying this, not making it more complicated.

> Our current assumption is that a CSE represents the "whole scene". Each CSE
> is effectively a different rendering of this whole.

[Duckworth, Mark] Yes

> With the above we are now
> changing that to imply that a CSE may be a part of a scene because which
> other Scenes/CSEs you chose is based on that rendering.

 [Duckworth, Mark] No, a CSE is still representing a whole scene.

> Its not a syntax issue only we're playing with the semantics of what we have
> also.

[Duckworth, Mark] We are suggesting adding syntax and semantics so a provider can give more suggestions to a consumer, trying to make the consumer's choice easier to make.  Not changing any existing semantics.

> >>>> I'm open to your idea of a single set of CSEs across all scenes,
> >>>> but I'm trying to understand if it will really be a scalable and
> >>>> less complex way to solve these issues.
> >>>
> >>> default devolves to just choosing the one best cse that works for
> >>> it, rather than having to choose from among combinations in
> >>> different scenes.
> >>>
> >>> I think that kind of thing is better done by email. (And I don't
> >>> have time to put together sufficient use cases before Tuesday.)
> >>>
> >>> More below.
> >>>
> >>>> Mark
> >>>>
> >>>>>>> ISTM that one should start out assuming that you need all
> >>>>>>> captures (that have encodings) in *some* CSE from *each* scene.
> >>>>> of, and we have been talking about at design team meetings, is
> >>>>> like
> >>>>> this:
> >>>>>> Scene 2 - previous active speaker Scene 3 - previous previous
> >>>>>> active speaker and so on...
> >>>>>
> >>>>> consumer has enough information to know this is a good choice if
> >>>>> it wants to show video from the current active speaker, including
> >>> together". But in a sense that is exactly the thing that standards
> >>> are working against.
> >>>
> >>> possible to construct a general purpose algorithm that will do as
> >>> well as algorithms that are special cased or that make "leaps of
> >>> faith" or use proprietary information.
> >>>
> >>>>> best user experience *if* the consumer has enough display
> >>>>> resources to show all these.
> >>>>
> >>>> think the consumer has enough information to make a good choice for
> >>>> a variety of different consumer capability levels in terms of
> >>>> number of decoders and displays it has.  Suppose the consumer can
> >>>> receive no more than 3 video encodings.  It could ask for the 3
> >>>> MCCs from Scene 1, or it could ask for 1 MCC from each scene.  But
> >>>> I would think it would do that only if each scene had a CSE with
> >>>> one MCC in it.  Each choice could give the "best" experience,
> >>>> depending on what the consumer is trying to present to the human
> users.
> >>>
> >>> worked out a lot of use cases, we could come close to agreeing to
> >>> what the "best experience" would be, in the absence of explicit
> >>> input from the end users in the room.
> >>>
> >>>>> active speaker" captures with highest priority, and the others
> >>>>> with descending priorities.
> >>>>
> >>>> [Duckworth, Mark] that's what I was thinking, but even that
> >>>> shouldn't be necessary because the consumer can make its own
> >>>> priority decision based on the policy attribute.
> >>>
> >>> decision. Lacking any understanding of the motivation for the
> >>> priority assignments, it is hard to trade those off against one another.
> >>>
> >>>>> Then, the consumer could start out assuming it should *try* for
> >>>>> one CSE from each scene. It can look at different combinations,
> >>>>> taking one from each scene, and evaluate what it can do with that
> >>>>> combination. For each
> >>>>> one:
> >>>>>
> >>>>> it can assign a score, depending on the priorities of those it
> >>>>> took, the priorities of those it had to abandon, and how much it
> >>>>> had to "adjust"
> >>>>> to work this out in detail for some specific consumer
> >>>>> configurations and see if it can actually find good
> >>>>> renderings.)
> >>>>>
> >>>>> Having the advertisement provide a single set of CSEs across all
> >>>>> the scenes would "help" in that it would reduce the number of
> >>>> Christian said, "The combinatorial issue becomes worse with the
> >>>> more Scenes that you have".  I'm concerned that trying to add
> >>>> another layer of alternatives in the advertisement, that crosses
> >>>> scene boundaries, won't scale well.  Or do you think it is really
> >>>> simpler than that?  Are you envisioning the single set of CSEs
> >>>> could be kept small, even with many scenes?
> >>>
> >>> With separate CSE lists in each scene, the *consumer* has to
> >>> evaluate all the combinations of one CSE from each scene.
> >>>
> >>> With a single CSE list, the *advertiser* has to consider all of
> >>> those when constructing the advertisement. But it doesn't have to
> >>> *include* them all in the advertisement. My gut tells me that when
> >>> there are only a few then perhaps they are all interesting, but when
> >>> they are many, that only a few will be interesting enough to
> >>> include. The advertiser is pruning the choices, using information
> >>> that it has that isn't necessarily available to the consumer.
> >>>
> >>> If the advertiser is typically going to include all the
> >>> combinations, then going this way is a bad choice. And if the
> >>> advertiser is going to limit what it includes because it is too much
> >>> work, or makes the advertisement too big, even though it thinks the
> >>> ones it is omitting would be useful, then this is also a bad idea.
> >>>
> >>>>> where I thought the media capture priority attribute should be
> >>>>> used.  I thought this was the purpose of adding the priority
> >>>>> attribute in the first place.  From the framework:
> >>>>>>
> >>>>>> 7.1.1.12. Priority
> >>>>> Captures.  The Provider sets this priority, and the Consumer MAY
> >>>>> use the priority to help decide which captures it wishes to
> >>>>> receive.
> >>>>> endpoint to determine which presentation is more important.
> >>>>> Priority is assigned at the individual capture level. It
> >>>>> represents the Provider's view of the relative priority between
> >>>>> Captures with a priority.
> >>>>>>
> >>>>> But consider, if prioritization were sufficient, we wouldn't need
> >>>>> to have CSEs at all. We could simply have prioritized captures.
> >>>>> ISTM that the reasons that CSEs are more powerful than priorities
> >>>>> within a scene should still apply outside a scene.
> >>>>
> >>>>> As I described above, I *think* priorities plus per-scene cse
> >>>>> lists can solve your use case. But the processing to use it that
> >>>>> way is complex.
> >>>>
> >>> I think it is less complex for the consumer to evaluate a single
> >>> list than to evaluate combinations of choices from multiple lists.
> >>> (Enumerating all the combinations and generating an equivalent
> >>> single list to evaluate is of course a straightforward programming
> >>> job. But I wonder if some implementers might instead go for "special
> >>> casing".)
> >>>
> >>>>>>>>> consumer should do when receiving an advertisement with
> >>>>>>>>> multiple scenes in order to get a sufficient representation of
> >>>>>>>>> the advertised
> >>>>>>> content.
> >>>>>>>>>
> >>>>>>>>> that case, one should take something from each scene
> >>>>>>>>> (typically one CSE from
> >>>>>>>>> each) to get "enough".
> >>>>>>>>>
> >>>>>>>>> In a "simple" case with an MCU use MCC, it might "pass
> >>>>>>>>> through" all the advertisements it receives, for their spatial
> >>>>>>>>> info, but not provide any encodings for them. Rather, it would
> >>>>>>>>> provide one or more new scenes using MCC to aggregate those
> >>>>>>>>> sources. Then one would not want to configure from the "pass
> >>>>>>>>> through" scenes, but that isn't an option anyway if there are
> >>>>>>>>> no encodings for them.
> >>>>>>>> consumer could use any capture attribute (not just spatial
> >>>>>>>> ones) to determine what media it wanted.
> >>>>>>>
> >>>>>>>>> relationship to one another. Some of those examples end up
> >>>>>>>>> with multiple scenes that do have encodings, where it doesn't
> >>>>>>>>> make sense to configure something from each scene.
> >>>>>>>> [CNG] I don't think there's anything in the framework that says
> >>>>>>>> a consumer must choose a capture from a scene.
> >>>>>>>
> >>>>>>> The question is how the consumer knows what is required to have
> >>>>>>> a "complete" representation of what is in the advertisement. It
> >>>>>>> may not want all that, but it is hard to make a good choice
> >>>>>>> without knowing.
> >>>>>>>
> >>>>>>> That is the point of CSEs - for the advertiser to indicate what
> >>>>>>> it considers to be good alternative choices. But CSEs only work
> >>>>>>> ISTM that one should start out assuming that you need all
> >>>>>>> captures (that have encodings) in *some* CSE from *each* scene.
> >>>>>>> You can then prune that based on the logic above. But is that
> >>>>>>> *enough* to make good
> >>>>> decisions?
> >>>>>>>>> Today we discussed using priority to solve this, but this
> >>>>>>>>> isn't really a priority problem, it is a problem of
> >>>>>>>>> *alternatives* that may be equally desirable, depending on the
> >>>>>>>>> resources or desires of the
> >>>>>>> consumer.
> >>>>>>>> [CNG] Perhaps you could give some examples? Does the list need
> >>>>>>>> to be exhaustive?
> >>>>>>>
> >>>>>>> I think each entry should be "sufficient" for somebody that
> >>>>>>> can't handle a bigger one. The definition of "sufficient" is
> subjective.
> >>>>>>>
> >>>>>>>>> Another possibility would be to leave CSEs as they are, but
> >>>>>>>>> add something analogous to a CSE list, but that lists
> >>>>>>>>> recommended combinations of scenes.
> >>>>>>>>>
> >>>>>>>>>       Thanks,
> >>>>>>>>>       Paul
> >>>>>>
> >>>>
> >>>>
> >>>