Subject: [clue] draft-duckworth-clue-switching-example-01

From: Paul Kyzivat <[email-masked]>
Date: Mon, 22 Jul 2013 12:49:46 -0400

Section 3.2 (Advertising multiple scenes):

While I understand what you are trying to do, I can't figure out how to 
make this approach work in a sensible way.

What logic would the MCU use to assign actual captures to the switched 
captures in the multiple scenes?

It could use site switching: the site of the most recent speaker in 
scene 1, 2nd most recent speaker in Scene 2, etc. As long as each site 
has three cameras that would work fine. But in the given example that 
isn't so. If the 2nd most recently speaking site only has one capture, 
and the 3rd most recently speaking site has three captures, does it 
leave VC5 and VC6 empty? That doesn't seem like an ideal choice.

Or does it back fill? (Put the most recently speaking four sites into 
Scenes 1...4. Then put the 5th most recently speaking site into the 
first captures in any of the sites that are still open, etc.) This would 
*work*, but it would violate the idea that the scenes are in priority 
order. So a receiving site that couldn't take them all might get some 
lower priority captures while missing some higher priority ones.

Section 4:

A consumer that can handle all 12 captures isn't the interesting one to 
consider here. More interesting are endpoints E,F,G. Lets consider F:

It can handle 8 captures: two big and 6 small. Or maybe one big and 12 
small.

The quality of the result depends upon the interaction between how the 
advertiser chooses to distribute sites among its advertised captures and 
the choices the consumer makes to distribute those among displays. And 
it doesn't look like there is enough information to allow the consumer 
to do the best possible job.

But I think we need to work the examples to see this.

Section 4.1 (One big scene):

    Issue: If the single screen endpoint wants to show 1 large image and
    three PiPs, then it must ask to receive 4 captures.  But how could it
    ask for one capture that represents the whole scene, plus 3 others
    that are additional lower priority, and that the 3 others shouldn't
    have a spatial relationship with the one large one?  The "Area of
    Display" idea would solve this issue.  A 2 screen consumer would be
    similar to a single screen endpoint, regarding this issue.

I think your example doesn't support this case. I thought the captures, 
while switched, each represented a single camera. I'm not sure what kind 
of advertisement would cover what you are questioning.

	Thanks,
	Paul

---

From: "Duckworth, Mark" <[email-masked]>
Date: Wed, 24 Jul 2013 13:11:46 -0700

Hi Paul,

Thanks very much for the comments!

Discussion inline.

Mark

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Paul Kyzivat
> Sent: Monday, July 22, 2013 12:50 PM
> To: CLUE
> Subject: [clue] draft-duckworth-clue-switching-example-01
> 
> While I understand what you are trying to do, I can't figure out how to make
> this approach work in a sensible way.
> 

[Duckworth, Mark] I see it as an extension of what the MCU could do if there was only one scene with a CSE with 3 captures for site switching (just to use an example).  If the most recent talker has 3 cameras, then use those.  If the most recent talker has just 1 camera, then what does the MCU do?  I think the question is fundamentally the same whether there is one simple scene with 3 captures, one scene with 12 captures in a CSE, or several scenes each with multiple captures.  I think a sensible thing for the MCU to do is to choose captures form other sites, maybe only the single camera sites, to fill in the rest of the switched captures that are not used by the most recent talker.

> It could use site switching: the site of the most recent speaker in scene 1, 2nd
> most recent speaker in Scene 2, etc. As long as each site has three cameras
> that would work fine. But in the given example that isn't so. If the 2nd most
> recently speaking site only has one capture, and the 3rd most recently
> speaking site has three captures, does it leave VC5 and VC6 empty? That
> doesn't seem like an ideal choice.

[Duckworth, Mark] I expect it would not leave VC5 and VC6 empty.

> Or does it back fill? (Put the most recently speaking four sites into Scenes
> 1...4. Then put the 5th most recently speaking site into the first captures in
> any of the sites that are still open, etc.) This would *work*, but it would
> violate the idea that the scenes are in priority order. 

[Duckworth, Mark] Yes, that sounds the same as what I was suggesting above.

> So a receiving site that
> couldn't take them all might get some lower priority captures while missing
> some higher priority ones.

[Duckworth, Mark] No, the consumer is getting the captures the consumer asked for.  The priority of those switched captures does not change. The MCU in this case is deciding which streams belong in the switched captures.  The priority of those switched captures is defined by the MCU.  So the MCU is the one that sets the priority and makes the decisions based on local policy.

[Duckworth, Mark] Again, I think this issue of how the MCU decides what to put in switched captures is really the same issue no matter how many capture scenes there are, how many captures in a CSE, or what the priority attribute values are.  If you think this issue is not really an issue if there is just one capture scene, can you explain why?

> It can handle 8 captures: two big and 6 small. Or maybe one big and 12 small.
> 
> advertiser chooses to distribute sites among its advertised captures and the
> choices the consumer makes to distribute those among displays.

[Duckworth, Mark] Yes, exactly.  draft-hansen-clue-consumer-layout and draft-pepperell-clue-switched-attribute suggest a solution for this.  That's why I'm suggesting it again.

> And it doesn't look like there is enough information to allow the consumer to do
> the best possible job.

[Duckworth, Mark] I think "the best possible job" is very subjective and can't be explicitly defined.  Different people will have different opinions about what is "best".  At this point I'm trying to make sure there is a way to do what most people would consider a "reasonably good job".

> I think your example doesn't support this case. I thought the captures, while
> switched, each represented a single camera. I'm not sure what kind of
> advertisement would cover what you are questioning.

[Duckworth, Mark] I don't follow you, but I'll try to explain my thoughts better.  Yes, each switched capture represents a single camera at a time, but which camera that is can be switched fairly frequently by the MCU.  Let's start with the single screen consumer that wants 1 large image to represent the highest priority, plus 3 small images of lower priority, and these 3 small images can be spatially related.  If the advertisement is the one in section 3.2, then the consumer would ask for VC1, VC4, VC5, VC6.  With VC1 being large (high resolution) and the rest small.  I think this would work well.  Similarly, a 2 screen consumer would ask for VC1, VC2, VC4, VC5, VC6, and maybe also VC7, VC8, VC9.  A very simple single screen consumer would ask for VC1.

[Duckworth, Mark] But if the advertisement is the one in section 3.1, then I don't know how the consumer could ask for what it wants.  This is where draft-hansen-clue-consumer-layout offers a solution.

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 25 Jul 2013 12:04:14 -0400

Mark,

More inline. There are some nasty problems here. This is hard!

> [Duckworth, Mark] I expect it would not leave VC5 and VC6 empty.

I agree.

> [Duckworth, Mark] No, the consumer is getting the captures the consumer asked for.  The priority of those switched captures does not change. The MCU in this case is deciding which streams belong in…

Maybe we are thinking of this differently.

My thinking is that the definition of the content of each capture in the 
advertisement is fixed at the time the advertisement is sent. So the 
configuration of which captures are to be received does not alter that.

Of course that is tricky for switched captures, especially 
site-switched. I think it can only be defined by an algorithm.
But it could be defined by an algorithm such as the one I described above.

(I'm not implying that the algorithm needs to be standardized, or even 
advertised, except in the most general terms.)

And of course the captures have their priorities fixed in the 
advertisement. So we presume that those with higher priority are more 
important to receive.

I'll also presume that more recent speakers, (or sites containing more 
recent speakers) are more important to receive that less recent ones. So 
ideally the captures containing the more recent ones should be the 
captures with the highest priority.

But trying to preserve spatial relationships seems to result in 
assigning some sites into captures with lower priority that their 
desired priority order.

I *think* you are suggesting that this would be fixed by doing the 
switching assignment to captures based on what captures have been 
configured, rather than only based on the advertisement.

Now maybe that is ok. But it makes it a real challenge to come up with a 
good definition of what a switched capture is, that makes it possible to 
make a good choice of what to configure.

> [Duckworth, Mark] Again, I think this issue of how the MCU decides what to put in switched captures is really the same issue no matter how many capture scenes there are, how many captures in a CSE,…

The constraints on the spatial relationships in a single scene make it 
simpler, at the expense of doing a poorer job of preserving spatial 
relationships.

It may be that we have simply tried to tackle an intractable problem.
So far I haven't seen a solution proposed that can do a good job in 
cases where the rooms have significantly different arrangements.

> [Duckworth, Mark] Yes, exactly.  draft-hansen-clue-consumer-layout and draft-pepperell-clue-switched-attribute suggest a solution for this.  That's why I'm suggesting it again.

Yes. The essence of these is that the consumer describes the rendering 
capabilities and the provider then chooses what to send that will best 
portray what the advertiser has available.

This is certainly a possible approach. But IMO it is a radical departure 
from the current model where the advertiser describes what is available, 
and the consumer chooses what it wants and how to best render what it 
has chosen.

I could see choosing either extreme. But it gets messy if we choose some 
halfway point between the two extremes. And that is what seems to have 
been proposed. Again, this is *possible*, but it gets messy.

Either extreme has tradeoffs:
- for our current approach, we are limited by how much the consumer
   knows about what it is choosing. The advertiser knows more, but
   we have chosen to limit how much is make available to the consumer,
   in order to keep things simple.

- for the "provider makes it right" approach, the provider has all
   the info about the sources. But then it is limited by what it knows
   about the consumer. The details about this haven't been worked out
   in great detail. But I think they will be difficult to describe
   thoroughly.

In any case, it seems to me that this is a difficult and important 
subject. If we get it wrong the result may be that the CLUE is found to 
be unusable.

> [Duckworth, Mark] I think "the best possible job" is very subjective and can't be explicitly defined.  Different people will have different opinions about what is "best".  At this point I'm trying…

Agreed. Perhaps we should set some metrics for this. E.g.,

- that it should produce the ideal result when all sites are identical
- set low, but attainable, goals for radically dissimilar sites.

> [Duckworth, Mark] I don't follow you, but I'll try to explain my thoughts better.  Yes, each switched capture represents a single camera at a time, but which camera that is can be switched fairly frequently by the MCU.  Let's start with the single screen consumer that wants 1 large image to represent the highest priority, plus 3 small images of lower priority, and these 3 small images can be spatially related.  If the advertisement is the one in section 3.2, then the consumer would ask for VC1, VC4, VC5, VC6.  With VC1 being large (high resolution) and the rest small.  I think 
this would work well.

This goes back to my question about whether the mappings are fixed with 
the advertisement, or dependent upon what is configured.

I would expect that VC1 would contain the current talker. If the 2nd 
most recent talker was from a different site with three cameras, then 
VC4..6 would contain that site, which would be ok.

But if the current talker came from a site with only one camera, and the 
2nd most recent talker came from a site with 1 or 2 cameras, then it is 
unclear that the 2nd most recent talking site will be mapped to VC4..6 - 
it might be mapped to VC2..3.

So the mapping algorithm must be very well understood by both ends in 
order to for this selection to work well.

Again, this just points to a need be very precise about what we mean by 
site switching.

> Similarly, a 2 screen consumer would ask for VC1, VC2, VC4, VC5, VC6, and maybe also VC7, VC8, VC9.  A very simple single screen consumer would ask for VC1.

I'm not going to discuss this case until we understand the prior one.

> [Duckworth, Mark] But if the advertisement is the one in section 3.1, then I don't know how the consumer could ask for what it wants.  This is where draft-hansen-clue-consumer-layout offers a solut…

Agreed.

	Thanks,
	Paul

---

From: "Duckworth, Mark" <[email-masked]>
Date: Thu, 25 Jul 2013 12:06:10 -0700

Hi Paul,

Yeah, it's hard, but I think we can come to a "good enough" solution to cover basic switched capture use cases.  It seems to me we are thinking along the same lines, for the most part, in the discussion below.  Particularly see my suggestion near the end, to see if it avoids some issues you raise.

Personally, I now think the scenarios I'm most interested in can be handled with the framework as is, by using the multiple scenes approach in section 3.2 of my draft.

I was hoping more people would be interested in discussing this on the list.

More inline.

Mark

> From: Paul Kyzivat [mailto:[email-masked]]
> Sent: Thursday, July 25, 2013 12:04 PM
> To: Duckworth, Mark
> Cc: CLUE
> Subject: Re: [clue] draft-duckworth-clue-switching-example-01
>
> Mark,
>
> >> From: [email-masked] [mailto:[email-masked]] On Behalf
> >> Of Paul Kyzivat
> >> While I understand what you are trying to do, I can't figure out how
> >> to make this approach work in a sensible way.
> >>
> >> What logic would the MCU use to assign actual captures to the
> >> switched captures in the multiple scenes?
> >
> > [Duckworth, Mark] I see it as an extension of what the MCU could do if
> there was only one scene with a CSE with 3 captures for site switching (just to
> use an example).  If the most recent talker has 3 cameras, then use those.  If
> the most recent talker has just 1 camera, then what does the MCU do?  I
> think the question is fundamentally the same whether there is one simple
> scene with 3 captures, one scene with 12 captures in a CSE, or several scenes
> each with multiple captures.  I think a sensible thing for the MCU to do is to
> choose captures form other sites, maybe only the single camera sites, to fill
> in the rest of the switched captures that are not used by the most recent
> talker.
> >
> >> scene 1, 2nd most recent speaker in Scene 2, etc. As long as each
> >> site has three cameras that would work fine. But in the given example
> >> that isn't so. If the 2nd most recently speaking site only has one
> >> capture, and the 3rd most recently speaking site has three captures,
> >> does it leave VC5 and VC6 empty? That doesn't seem like an ideal choice.
> >
> >> first captures in any of the sites that are still open, etc.) This
> >> would *work*, but it would violate the idea that the scenes are in priority
> order.
> >
> >> couldn't take them all might get some lower priority captures while
> >> missing some higher priority ones.
> >
> > [Duckworth, Mark] No, the consumer is getting the captures the consumer
> asked for.  The priority of those switched captures does not change. The
> MCU in this case is deciding which streams belong in the switched captures.
> The priority of those switched captures is defined by the MCU.  So the MCU
> is the one that sets the priority and makes the decisions based on local policy.
>

[Duckworth, Mark] I think it is okay if the Provider alters its algorithm for switching depending on what the consumer has configured.  I think this is particularly true for the case where the advertisement has a CSE attribute for "explicitly signaling that a subset of the constituent captures can be used to produce a valid representation of that scene."  We haven't agreed yet on adding such an attribute, maybe it is a bad idea, see below for another example and question about it.

> Of course that is tricky for switched captures, especially site-switched. I think
> it can only be defined by an algorithm.
> And of course the captures have their priorities fixed in the advertisement.
> So we presume that those with higher priority are more important to
> receive.
>
> I'll also presume that more recent speakers, (or sites containing more recent
> speakers) are more important to receive that less recent ones. So ideally the
> captures containing the more recent ones should be the captures with the
> highest priority.
>
> But trying to preserve spatial relationships seems to result in assigning some
> sites into captures with lower priority that their desired priority order.

[Duckworth, Mark] Yes, and I think that is okay.

> I *think* you are suggesting that this would be fixed by doing the switching
> assignment to captures based on what captures have been configured,
> rather than only based on the advertisement.

[Duckworth, Mark] Yes, I'm saying the actual switching algorithm used by the MCU could depend on which captures the consumer has configured.  I don't see any problem with that.

> Now maybe that is ok. But it makes it a real challenge to come up with a good
> definition of what a switched capture is, that makes it possible to make a
> good choice of what to configure.

[Duckworth, Mark] I think the definition of switching algorithm decisions has to be loose, to let the Provider do what it wants to do.  If the consumer wants to have more control, then the consumer should be making more specific decisions about which original captures it wants to receive, including changing that decision fairly frequently as current talkers change.  I think this is an interesting topic also, not specific to CLUE.

> > [Duckworth, Mark] Again, I think this issue of how the MCU decides what
> to put in switched captures is really the same issue no matter how many
> capture scenes there are, how many captures in a CSE, or what the priority
> attribute values are.  If you think this issue is not really an issue if there is just
> one capture scene, can you explain why?
>
> The constraints on the spatial relationships in a single scene make it simpler,
> at the expense of doing a poorer job of preserving spatial relationships.
>
> So far I haven't seen a solution proposed that can do a good job in cases
> where the rooms have significantly different arrangements.

[Duckworth, Mark] I think the multiple scenes idea from section 3.2 of my draft does a good job for cases I'm interested in, including the example in the draft.

> >> Section 4:
> >>
> >> A consumer that can handle all 12 captures isn't the interesting one
> >> to consider here. More interesting are endpoints E,F,G. Lets consider F:
> >>
> >> The quality of the result depends upon the interaction between how
> >> the advertiser chooses to distribute sites among its advertised
> >> captures and the choices the consumer makes to distribute those among
> displays.
> >
> > [Duckworth, Mark] Yes, exactly.  draft-hansen-clue-consumer-layout and
> draft-pepperell-clue-switched-attribute suggest a solution for this.  That's
> why I'm suggesting it again.
>
> capabilities and the provider then chooses what to send that will best portray
> what the advertiser has available.
>
> This is certainly a possible approach. But IMO it is a radical departure from the
> current model where the advertiser describes what is available, and the
> consumer chooses what it wants and how to best render what it has chosen.

[Duckworth, Mark] Yes, that's one reason why I'm leaning toward using the multiple scenes approach, because it works with the current framework.

>    in order to keep things simple.

[Duckworth, Mark] Yes.  I think this is okay.

> In any case, it seems to me that this is a difficult and important subject. If we
> get it wrong the result may be that the CLUE is found to be unusable.

[Duckworth, Mark] I still think the multiple scenes advertisement works, along with the "provider makes it right" approach as you say.

> >> And it doesn't look like there is enough information to allow the
> >> consumer to do the best possible job.
> >
> > [Duckworth, Mark] I think "the best possible job" is very subjective and
> can't be explicitly defined.  Different people will have different opinions
> about what is "best".  At this point I'm trying to make sure there is a way to
> do what most people would consider a "reasonably good job".
>
> >> I think your example doesn't support this case. I thought the
> >> captures, while switched, each represented a single camera. I'm not
> >> sure what kind of advertisement would cover what you are questioning.
> >
> > [Duckworth, Mark] I don't follow you, but I'll try to explain my
> > thoughts better.  Yes, each switched capture represents a single
> > camera at a time, but which camera that is can be switched fairly
> > frequently by the MCU.  Let's start with the single screen consumer
> > that wants 1 large image to represent the highest priority, plus 3
> > small images of lower priority, and these 3 small images can be
> > spatially related.  If the advertisement is the one in section 3.2,
> > then the consumer would ask for VC1, VC4, VC5, VC6.  With VC1 being
> > large (high resolution) and the rest small.  I think
> this would work well.
>
> I would expect that VC1 would contain the current talker. If the 2nd most
> recent talker was from a different site with three cameras, then
> But if the current talker came from a site with only one camera, and the 2nd
> most recent talker came from a site with 1 or 2 cameras, then it is unclear
> that the 2nd most recent talking site will be mapped to VC4..6 - it might be
> mapped to VC2..3.

[Duckworth, Mark] Yes, could be either, depends on the algorithm/policy in the provider.  Consumer doesn't need to have knowledge of this.

> order to for this selection to work well.

[Duckworth, Mark] I disagree.  I am thinking of "the provider makes it right" approach, as you put it.  In my view, the consumer doesn't need to know any more details of the provider's switching algorithm.

[Duckworth, Mark] Maybe the concept of "explicitly signaling that a subset of the constituent captures can be used to produce a valid representation of that scene" is making this more complicated than it needs to be.  Suppose the advertisement was like this, where all the captures are switched:

Scene 1: (higher priority captures)
  CSE1 (VC1, VC2, VC3)
  CSE2 (VC4, VC5)
  CSE3 (VC6)
Scene 2: (lower priority captures)
  CSE1 (VC7, VC8, VC9)
  CSE2 (VC10, VC11)
  CSE3 (VC12)
and so on...

Now consider the single screen consumer that wants 1 large image to represent the highest priority, plus 3 small images of lower priority, and these 3 small images can be spatially related.  This consumer will ask for VC6 (high resolution), VC7, VC8, VC9 (all low res).  In this case, we can distinguish between VC6 and VC1.  It is more clear now that VC6 should represent the entire scene 1.  With scenes like this, it seems the switching algorithm is less dependent (maybe not at all?) on what has been configured by the consumer.  Does this approach avoid some issues you raise?

> Again, this just points to a need be very precise about what we mean by site
> switching.

[Duckworth, Mark] I think it should be very loose, doesn't need to be precise at all.

> > Similarly, a 2 screen consumer would ask for VC1, VC2, VC4, VC5, VC6, and
> maybe also VC7, VC8, VC9.  A very simple single screen consumer would ask
> for VC1.
>
> > [Duckworth, Mark] But if the advertisement is the one in section 3.1, then I
> don't know how the consumer could ask for what it wants.  This is where
> draft-hansen-clue-consumer-layout offers a solution.
>
> Agreed.
>
>       Thanks,
>       Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 25 Jul 2013 17:28:16 -0400

> I was hoping more people would be interested in discussing this on the list.

Me too. But we are now into the period when many people are in transit 
or otherwise very busy. So we can't infer anything from silence right now.

I am starting to think that we might want/need to form a group of people 
who are both knowledgeable and interested in this subject, to pursue it 
in depth. Ensuring that reasonable choices can be made, especially in 
common cases, but even in hard cases, will take a lot of thought I 
think. (And this needs to work for audio as well as video!)

It will not be nice to find out after the fact that implementation A 
can't work in a reasonable way with implementation B because of 
inadequacies here.

More inline.

> [Duckworth, Mark] Yes, and I think that is okay.

IMO its not ok if it results in displaying less important stuff while 
omitting more important stuff.

> [Duckworth, Mark] I think the definition of switching algorithm decisions has to be loose, to let the Provider do what it wants to do.  If the consumer wants to have more control, then the consumer…

I'm not trying to suggest that the exact switching algorithms need to be 
specified.

But the consumer needs to have some expectations about some properties 
of the switching algorithms when choosing which switched captures to 
configure.

If there are *no* guarantees, then if there are three switched captures 
and I only take one of them, then I might get something totally useless.

And consider if I must choose one of three switched video captures, and 
one of three switched audio captures. What guarantee do I have that they 
go together?

The algorithm I construct to decide which captures to configure and how 
to map them onto local equipment only has a few things to work with:
- the available equipment for rendering the received captures
- the advertisement
- the guarantees (whichever ones we include) in the clue spec about
   how switching works.

Those guarantees are an important part of that.

> [Duckworth, Mark] I think the multiple scenes idea from section 3.2 of my draft does a good job for cases I'm interested in, including the example in the draft.

You think it works for this case, given your assumptions about how 
provider makes it right works. I'm not convinced yet. I'll give it a 
"maybe".

> [Duckworth, Mark] Yes, that's one reason why I'm leaning toward using the multiple scenes approach, because it works with the current framework.

IIUC what you are talking about is a point between the two extremes.

> [Duckworth, Mark] I still think the multiple scenes advertisement works, along with the "provider makes it right" approach as you say.

So this is mostly of the "advertiser specifies fixed choices and 
consumer chooses", but with a bit of "provider makes it right". But the 
provider in this case has very limited data to help it decide what is 
right. Specifically, the only thing it manipulates is the switching 
algorithm, and the only data it has to work with to choose is which of 
the switched captures have been configured.

I am having trouble getting my head around the possibilities and 
limitations that are present in that context.

> [Duckworth, Mark] Yes, could be either, depends on the algorithm/policy in the provider.  Consumer doesn't need to have knowledge of this.

I think the consumer wants to know that mapping will be a good one, not 
a bad one.

> [Duckworth, Mark] I disagree.  I am thinking of "the provider makes it right" approach, as you put it.  In my view, the consumer doesn't need to know any more details of the provider's switching al…

I commented on this one above.

> Scene 2: (lower priority captures)

I think I want to know the relative priorities to understand.
I'm thinking you must mean something like:

Scene 1: (higher priority captures)
   CSE1 (VC1:100, VC2:101, VC3:102)
   CSE2 (VC4:100, VC5:101)
   CSE3 (VC6:100)
Scene 2: (lower priority captures)
   CSE1 (VC7:200, VC8:201, VC9:202)
   CSE2 (VC10:200, VC11:201)
   CSE3 (VC12:200)

The above assumes lower numbers are higher priorities.
I'm not confident in the above. I think using disjoint priority ranges 
for Scene 1 and Scene 2 is right. But I'm not sure what to do with 
priorities of the captures in a single scene.

Obviously this interacts with how the consumer uses the priorities to 
assign captures to rendering devices.

So what is the algorithm in this case? I can think of two fundamentally 
different ways to do that: either work through the available rendering 
devices and choose a capture for each, or else work through the captures 
in priority order and try to find rendering devices to put them on.

I was going to work through the case now using one of those techniques. 
But it immediately presented complications. Clearly the algorithm is 
going to be tricky, and I couldn't come up with anything I wanted to 
write here.

So I'm not sure if my assignment of priorities is helpful or not.
It is bad if the advertiser can only know how to assign priorities by 
knowing the algorithm the consumer will use for selection.

> Now consider the single screen consumer that wants 1 large image to represent the highest priority, plus 3 small images of lower priority, and these 3 small images can be spatially related.  This consumer will ask for VC6 (high resolution), VC7, VC8, VC9 (all low res).

Are you assuming that VC6 is only offered in high res, while VC7..9 are 
only offered in low res? Or are you assuming that all are offered at 
either res? If each is only offered in one res, then why?

> In this case, we can distinguish between VC6 and VC1.

Its clear that if you want to represent the scene with one capture then 
VC6 is the better choice, while if you want to represent the scene with 
three captures then VC1..3 is a better choice.

But since it is *possible* to render scene 1 with one capture, it isn't 
clear if it is better to do that and save the other devices for other 
scenes, or if its better to devote more devices to scene 1 and fewer to 
the other scenes.

What is hard about this is to ensure, when thinking about specific 
cases, that the assumption about how the advertisement is constructed 
isn't biased by knowledge about characteristics of the consumer that a 
real advertiser won't know. And conversely when considering the consumer 
behavior, that it isn't drawing inferences about what the advertisement 
means that really depend on knowledge about how the advertiser works.

>  It is more clear now that VC6 should represent the entire scene 1.  With scenes like this, it seems the switching algorithm is less dependent (maybe not at all?) on what has been configured by the consumer.  Does this approach avoid some issues you raise?
>

Again, see earlier comment.

	Thanks,
	Paul