Subject: Re: [clue] Capture scene clarifications

From: "Duckworth, Mark" <[email-masked]>
Date: Sat, 1 Dec 2012 07:18:58 -0800

Points a) and b) are consistent with the intent of the current framework.  Point d) starts to change the intent.  The framework allows for a media consumer to receive multiple capture scene entries (regardless of media type) at the same time.  I think point d) is proposing we not allow this.  I think saying "A consumer shall choose a capture scene entry rather than choosing individual captures from multiple entries" is equivalent to saying "A consumer shall choose at most one capture scene entry (of a particular media type) from any particular capture scene".

Such a restriction probably reduces the scenarios in which the provider needs to express simultaneous transmission sets, because some of the simultaneity constraints are now included in the capture scene entries.  As others have pointed out, there is still the possibility of simultaneous constraints that apply across multiple capture scenes.

Adding this restriction, to not allow multiple capture scene entries (of same media type, in the same scene) to be used simultaneously is okay with me, but I don't think it really simplifies much.

As others have said, allowing a consumer to choose a subset of captures from a capture scene entry makes sense to me too.

I have a few more comments about the proposals inline below.

Mark

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Christian Groves
> Sent: Thursday, November 22, 2012 10:56 PM
> To: [email-masked]
> Subject: [clue] Capture scene clarifications
> 
> 
> Hello,
> 
> Paul K reminded me that I have an action point from the interim meeting
> regarding my draft <draft-groves-clue-scene-clarifications>. At the meeting it
> seems that there was general agreement with the intent of the
> proposals:
>     a)      A scene represents an area where the capture devices are
>             spatially related, i.e. a presentation that shares no spatial
>             relation with a video is a separate scene.
> 
>     b)      A capture scene entry thus represents alternate
>             representations of a complete scene.  The provider is the one
>             that determines what a complete scene is.  A consumer choses
>             a capture scene entry from the scene in the knowledge that it
>             represents the entire scene.
> 
>     d)      A consumer shall chose a capture scene entry rather than
>             choosing individual captures from multiple entries.  This
>             does not mean that an consuming endpoint must render all the
>             captures.  What is locally rendered and how is a local
>             decision.
> 
> I think the framework draft could be enhanced in a few places to make the
> intent of CLUE clearer. Below are some suggested updates.
> 
> 1) Introduce a new definition for "Scene". The capture scene definition and
> several places mention "scene" but there's no explanation. I propose the
> following definition to be added to section 3 of the draft:
> Scene: Represents an area where the capture devices are spatially related.
> Non-spatially related devices exist in different scenes.

[Duckworth, Mark] Sounds good.

> 2) Update the definition of "Capture scene entry" to make it clearer that it
> represents an entire scene. Proposed text is:
> *Capture Scene Entry: a list of media captures of the same media type
>     that together form one representation of an entire capture scene.

[Duckworth, Mark] Sounds good.

> 3) Update section 6.2 on Capture scenes to reflect the above intents.
> Changes proposals are:
> - Section 6.2 2nd paragraph:
> "A capture scene is a structure representing the <<entire>> scene that is
>     captured by a collection of<<spatially related>>capture devices.  A capture
> scene..."

 [Duckworth, Mark] Sounds good. 

> - Section 6.2 3rd paragraph:
>   "A provider may advertise multiple capture scenes or just a single
>     capture scene.<<What constitutes an entire scene is up to the provider.>>
> A media provider might typically use one capture
>     scene for main participant media and another capture scene for a
>     computer generated presentation...."

 [Duckworth, Mark] sounds good

> - Section 6.2 4th paragraph:
> "A media provider arranges media captures in a capture scene to help
>     the media consumer choose which captures it wants.  The capture scene
>     entries in a capture scene are different alternatives the provider is
>     suggesting for representing the entire capture scene.<delete rest of
> paragraph>".
> 
> - Section 6.2 5th paragraph:
>   "Media captures within the same capture scene entry must be of the
>     same media type - it is not possible to mix audio and video captures
>     in the same capture scene entry, for instance.  The provider must be
>     capable of encoding and sending all media captures in a single entry
>     simultaneously."  <delete rest of paragraph>
> 
> - section 6.2 New paragraph dealing with consumer behaviour after the 7th
> paragraph.
>   "A consumer may receive an advertisement with multiple capture scenes.
> A consumer may choose to receive any number of capture scenes. An
> advertised capture scene it may contain one of more capture scene entries
> that may contain one of more media captures. A consumer may choose an
> capture scene entry in the knowledge that it is a complete representation of
> the scene for a particular media type and that all media captures are spatially
> related. For a particular media type the consumer shall choose one capture
> scene entry rather than choosing individual captures from multiple capture
> scene entries. However this does not mean that a consuming endpoint must
> render all the media captures. What is locally rendered and how is a local
> decision."

[Duckworth, Mark] These three proposed changes above are related to not allowing multiple capture scene entries, of the same media type, in the same scene, from being used simultaneously.  I could go either way on this change, whatever the group decides.

> 4) The framework uses the terms "media provider" and "provider". We
> probably should be consistent in the use of the term throughout the
> framework.

[Duckworth, Mark] sounds good.

> 5) Related to the "capture group" concept that didn't seem to get support is
> the ordering in the Capture scene, Capture scene entry, Media captures.
> There was some discussion that people had assumed some sort of
> prioritisation / ordering related to these structures. i.e. If I have a capture
> scene entry VC0, VC1, VC2 do I assume they should be rendered left to right
> or is there nothing to be assumed? I take it is the later understanding put I
> didn't see anything in the framework regarding what should be assumed. We
> probably should also add something for that.

[Duckworth, Mark] From the framework: "Determination of the order of these captures (VC0, VC1 and VC2) for rendering purposes is accomplished through use of their Area of Capture attributes."  I don't know what other type or order or prioritization you might be talking about here.

> Thoughts?
> 
> Regards, Christian
> _______________________________________________
> clue mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/clue

---

From: Christian Groves <[email-masked]>
Date: Mon, 03 Dec 2012 15:04:48 +1100

Hello Mark,

I'm no strongly against being able to select individual captures however 
it seems to me to add complications and I think that if we were to 
continue to go in that direction could benefit from further discussion 
in the framework.

What isn't clear to me in the current framework is whether the sending 
of simultaneous transmission sets is mandatory and if not what omitting 
this part of an advertisement means?

Please see some further replies below.

Regards, Christian

> Such a restriction probably reduces the scenarios in which the provider needs to express simultaneous transmission sets, because some of the simultaneity constraints are now included in the capture…
[CNG] I don't think point d) changes this. The current description of 
capture scene entry says that a provider must be able to send all the 
media associated with the individual captures within a capture scene 
entry concurrently. This would also need to be reflected in any 
simultaneous set entries. This seems to be a place for possible 
inconsistencies because the data would effectively be duplicated.
> [Duckworth, Mark] From the framework: "Determination of the order of these captures (VC0, VC1 and VC2) for rendering purposes is accomplished through use of their Area of Capture attributes."  I do…
[CNG] One aspect is the "spatial ordering" i.e. how something is 
rendered the other aspect is regarding how the consumer interprets the 
order of scene/capture scene entry/capture within the message. i.e. Is 
the first capture scene entry more important than the last entry? There 
has been some discussion previously that some people that the provider 
of the advertisement could provide some prioritisation hint to the 
consumer by ordering the list of capture scene entries. From the text 
you mentioned aspect is there a general conclusion that there is no 
prioritisation implied from the ordering of elements in the advertisement?

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 3 Dec 2012 07:53:17 +0000

Hi,

I think there are two questions here:

1) Is it allowed to choose multiple capture scene entries (associated with the same scene)?
2) Is it allowed to choose, from a given capture scene entry, a subset of captures?

Regarding 1), I see very little reasons for allowing it. Of course, it shall be possible to receive multiple capture scene entries, associated with *different* scenes. A typical example is one entry associated with the "room", and one associated with a "presentation".

Regarding 2), during the SDP offer/answer, nothing can prevent the answerer from rejecting streams associated with specific captures. But, I am not sure we need to cover it in the CLUE signaling. In the CLUE signaling the answerer indicates which capture scene entry it wants - including all captures associated with it.

Regards,

Christer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Christian Groves
Sent: 3. joulukuuta 2012 6:05
To: [email-masked]
Subject: Re: [clue] Capture scene clarifications

Hello Mark,

I'm no strongly against being able to select individual captures however it seems to me to add complications and I think that if we were to continue to go in that direction could benefit from further discussion in the framework.

What isn't clear to me in the current framework is whether the sending of simultaneous transmission sets is mandatory and if not what omitting this part of an advertisement means?

Regards, Christian

> Such a restriction probably reduces the scenarios in which the provider needs to express simultaneous transmission sets, because some of the simultaneity constraints are now included in the capture…
[CNG] I don't think point d) changes this. The current description of capture scene entry says that a provider must be able to send all the media associated with the individual captures within a capture scene entry concurrently. This would also need to be reflected in any simultaneous set entries. This seems to be a place for possible inconsistencies because the data would effectively be duplicated.
>> From: [email-masked] [mailto:[email-masked]] On Behalf 
>> Of Christian Groves
>> Paul K reminded me that I have an action point from the interim 
>> meeting regarding my draft <draft-groves-clue-scene-clarifications>. 
>> At the meeting it seems that there was general agreement with the 
>> intent of the
>> proposals:
>> I think the framework draft could be enhanced in a few places to make 
>> the intent of CLUE clearer. Below are some suggested updates.
>>
>> 1) Introduce a new definition for "Scene". The capture scene 
>> definition and several places mention "scene" but there's no 
>> explanation. I propose the following definition to be added to section 3 of the draft:
>> 2) Update the definition of "Capture scene entry" to make it clearer 
>> that it represents an entire scene. Proposed text is:
>>      captured by a collection of<<spatially related>>capture devices.  
>> A capture scene..."
>>      capture scene.<<What constitutes an entire scene is up to the 
>> provider.>> A media provider might typically use one capture
>> - section 6.2 New paragraph dealing with consumer behaviour after the 
>> 7th paragraph.
>> advertised capture scene it may contain one of more capture scene 
>> entries that may contain one of more media captures. A consumer may 
>> choose an capture scene entry in the knowledge that it is a complete 
>> representation of the scene for a particular media type and that all 
>> media captures are spatially related. For a particular media type the 
>> consumer shall choose one capture scene entry rather than choosing 
>> individual captures from multiple capture scene entries. However this 
>> does not mean that a consuming endpoint must render all the media 
>> captures. What is locally rendered and how is a local decision."
>> 5) Related to the "capture group" concept that didn't seem to get 
>> support is the ordering in the Capture scene, Capture scene entry, Media captures.
>> prioritisation / ordering related to these structures. i.e. If I have 
>> a capture scene entry VC0, VC1, VC2 do I assume they should be 
>> rendered left to right or is there nothing to be assumed? I take it 
>> is the later understanding put I didn't see anything in the framework 
>> regarding what should be assumed. We probably should also add something for that.
[CNG] One aspect is the "spatial ordering" i.e. how something is rendered the other aspect is regarding how the consumer interprets the order of scene/capture scene entry/capture within the message. i.e. Is the first capture scene entry more important than the last entry? There has been some discussion previously that some people that the provider of the advertisement could provide some prioritisation hint to the consumer by ordering the list of capture scene entries. From the text you mentioned aspect is there a general conclusion that there is no prioritisation implied from the ordering of elements in the advertisement?

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 03 Dec 2012 10:35:25 -0500

> Regarding 1), I see very little reasons for allowing it. Of course, it shall be possible to receive multiple capture scene entries, associated with *different* scenes. A typical example is one entr…

I can see how it could be useful for a MCU to do this.
This would give it the flexibility to offer a wider range of choices in 
its own advertisements.

	Thanks,
	Paul

---

From: "Duckworth, Mark" <[email-masked]>
Date: Mon, 3 Dec 2012 09:27:52 -0800

Hello Christian,

I agree the framework (or other document, after refactoring) needs to be clear about whether expressing simultaneous transmission sets is mandatory or not.  Maybe it should be mandatory for the media provider to express a simultaneous transmission set that includes all the media captures, even if it has no restrictions.

About the order of capture scenes, entries within a scene, and captures within an entry - the framework intent was that this order had no significance.  But it does seem it might be useful to use the ordering as a prioritization hint from provider to consumer.  If we decide we want to do this, we need to make it clear in the framework (or possibly lower level document, if that is where it ends up after refactoring).

Mark

> Sent: Sunday, December 02, 2012 11:05 PM
> To: [email-masked]
> I'm no strongly against being able to select individual captures however it
> seems to me to add complications and I think that if we were to continue to
> go in that direction could benefit from further discussion in the framework.
> 
> What isn't clear to me in the current framework is whether the sending of
> simultaneous transmission sets is mandatory and if not what omitting this
> part of an advertisement means?
> 
> > Points a) and b) are consistent with the intent of the current framework.
> Point d) starts to change the intent.  The framework allows for a media
> consumer to receive multiple capture scene entries (regardless of media
> type) at the same time.  I think point d) is proposing we not allow this.  I think
> saying "A consumer shall choose a capture scene entry rather than choosing
> individual captures from multiple entries" is equivalent to saying "A
> consumer shall choose at most one capture scene entry (of a particular
> media type) from any particular capture scene".
> >
> > Such a restriction probably reduces the scenarios in which the provider
> needs to express simultaneous transmission sets, because some of the
> simultaneity constraints are now included in the capture scene entries.  As
> others have pointed out, there is still the possibility of simultaneous
> constraints that apply across multiple capture scenes.
> [CNG] I don't think point d) changes this. The current description of capture
> scene entry says that a provider must be able to send all the media
> associated with the individual captures within a capture scene entry
> concurrently. This would also need to be reflected in any simultaneous set
> entries. This seems to be a place for possible inconsistencies because the
> data would effectively be duplicated.
> >
> > Adding this restriction, to not allow multiple capture scene entries (of same
> media type, in the same scene) to be used simultaneously is okay with me,
> but I don't think it really simplifies much.
> >
> > As others have said, allowing a consumer to choose a subset of captures
> from a capture scene entry makes sense to me too.
> >
> > [Duckworth, Mark] These three proposed changes above are related to not
> allowing multiple capture scene entries, of the same media type, in the same
> scene, from being used simultaneously.  I could go either way on this change,
> whatever the group decides.
> >
> > [Duckworth, Mark] From the framework: "Determination of the order of
> these captures (VC0, VC1 and VC2) for rendering purposes is accomplished
> through use of their Area of Capture attributes."  I don't know what other
> type or order or prioritization you might be talking about here.
> [CNG] One aspect is the "spatial ordering" i.e. how something is rendered
> the other aspect is regarding how the consumer interprets the order of
> scene/capture scene entry/capture within the message. i.e. Is the first
> capture scene entry more important than the last entry? There has been
> some discussion previously that some people that the provider of the
> advertisement could provide some prioritisation hint to the consumer by
> ordering the list of capture scene entries. From the text you mentioned
> aspect is there a general conclusion that there is no prioritisation implied
> from the ordering of elements in the advertisement?
> >
> >> Thoughts?
> >>
> >> Regards, Christian

---

From: Roberta Presta <[email-masked]>
Date: Mon, 03 Dec 2012 19:58:19 +0100

Hi all,

I have a couple of questions about the use of simultaneous sets.
I have a camera that can be used to capture (i) a zoomed-out view of the 
telepresence room (say VC1)  and a (ii) moving zoomed-in view of the 
current speaker (say VC2).
VC1 and VC2 can not clearly be sent simultaneously, then they never 
appear in the same simultaneous set.

Question nr1: If I listed all the possible simultaneous sets, should I 
have to provide all the possible combinations of captures without having 
together VC1 and VC2?

Question nr2: What I understand is that simultaneity constraints that 
would be expressed by using simultaneous sets are mainly related to the 
impossibility of sending simultaneously different captures that are 
captured by the same device.
Is it the only motivation because of which the simultaneous sets have 
been designed?
Indeed, other constraints related to the simultaneous sending of 
multiple captures can be deduced using the encodings bandwidth information.

If the answer to Question nr2 is "no", can you please provide other 
cases in which simultaneous sets are needed (Question nr3)? They are not 
reported in the framework document.
Otherwise, If the answer to Question nr2 is "yes", maybe there is an 
easier way to handle the issue.
We can think of providing a description of capture devices in a scene, 
and then link the captures to their capture device.
It goes straightforward that two captures of the same media type taken 
from the same capture device can not be sent simultaneously.
In other words, we can  convey information about which captures are 
mutually exclusive rather than provide possible alternatives of 
simultaneous captures.

Cheers,
Roberta

---

From: Andy Pepperell <[email-masked]>
Date: Tue, 4 Dec 2012 17:32:09 +0000

Hi Roberta,

We had 2 main use cases in mind when thinking about the simultaneity
restrictions of media captures.
- the first is the one you mention, multiple media captures that use the
same physical device with, say, different P/T/Z values, which cannot be
provided at the same time
- the second case was for something like a switched or transcoded MCU which
might provide 1, 2, 3 and 4 capture alternate views of a conference, but
might not want to be obliged to provide, say, the leftmost capture of 2 and
the centre capture of 3 simultaneously.

As part of the framework, we thought a little about the best way to express
these restrictions; the first case lends itself well to a small number of
small simultaneous transmission sets, which each set listing just the
captures that cannot be provided together. Our feeling was that listing all
of the sets which *could* be provided together could lead to a
combinatorial explosion as every pair of captures which had a restriction
would essentially double the number of possible sets that would need
listing out - we also felt that this would make consumer-side behavior
potentially more complex as the consumer would have to look through a large
number of sets and couldn't assume that those sets were just a long hand
version of a few [typically] binary choices.

>Roberta: We can think of providing a description of capture devices in a
scene, and then link the captures to their capture device.
>Roberta: In other words, we can  convey information about which captures
are mutually exclusive rather than provide possible alternatives of
simultaneous captures.

Yes, I think all of the above is basically agreeing with your points,
though we thought it useful to not introduce the notion of physical devices
and the mapping of the more abstract capture devices onto them, as there
are lots of cases where there might not be such physical devices (e.g. an
MCU, or playing back from a recording). To my thinking, all of the benefit
of this approach is achieved by having the abstract sets of captures which
cannot be provided together, with the additional benefit that systems
sufficiently simple to have no such restrictions could just omit these sets
completely.

The hypothesised MCU case above where 1, 2, 3 and 4 capture versions of the
same scene were desired not to be provided together is more problematic
with this idea though, and that case would seem to work better in an
environment where the sets of captures that *could* be provided together
were listed, instead of sets of captures that could not. However, we felt
that this was likely to be less common...

Hope this helps,

Andy

On Mon, Dec 3, 2012 at 6:58 PM, Roberta Presta <[email-masked]>wrote:

> VC1 and VC2 can not clearly be sent simultaneously, then they never appear
> in the same simultaneous set.
>
> Is it the only motivation because of which the simultaneous sets have been
> designed?
> Indeed, other constraints related to the simultaneous sending of multiple
> captures can be deduced using the encodings bandwidth information.
>
> If the answer to Question nr2 is "no", can you please provide other cases
> in which simultaneous sets are needed (Question nr3)? They are not reported
> in the framework document.
> We can think of providing a description of capture devices in a scene, and
> then link the captures to their capture device.
>> I agree the framework (or other document, after refactoring) needs to be
>> clear about whether expressing simultaneous transmission sets is mandatory
>> or not.  Maybe it should be mandatory for the media provider to express a
>> simultaneous transmission set that includes all the media captures, even if
>> it has no restrictions.
>>
>>
>>
> ______________________________**_________________
> clue mailing list
> [email-masked]
> https://www.ietf.org/mailman/**listinfo/clue<https://www.ietf.org/mailman/listinfo/clue>
>

---

From: "Duckworth, Mark" <[email-masked]>
Date: Tue, 4 Dec 2012 10:41:49 -0800

We discussed at IETF83 the different ways of representing the simultaneous sets - either by what can be used simultaneously, or by what is mutually exclusive.  According to the minutes<http://tools.ietf.org/wg/clue/minutes?item=minutes-83-clue.html>, we didn't come to any consensus to change what was already in the framework (which is to use sets of what can be provided simultaneously).  This is the same as the current framework.

Mark

From: [email-masked] [mailto:[email-masked]] On Behalf Of Andy Pepperell
Sent: Tuesday, December 04, 2012 12:32 PM
To: Roberta Presta
Cc: [email-masked]
Subject: Re: [clue] simultaneous sets

Hi Roberta,

We had 2 main use cases in mind when thinking about the simultaneity restrictions of media captures.
- the first is the one you mention, multiple media captures that use the same physical device with, say, different P/T/Z values, which cannot be provided at the same time
- the second case was for something like a switched or transcoded MCU which might provide 1, 2, 3 and 4 capture alternate views of a conference, but might not want to be obliged to provide, say, the leftmost capture of 2 and the centre capture of 3 simultaneously.

As part of the framework, we thought a little about the best way to express these restrictions; the first case lends itself well to a small number of small simultaneous transmission sets, which each set listing just the captures that cannot be provided together. Our feeling was that listing all of the sets which *could* be provided together could lead to a combinatorial explosion as every pair of captures which had a restriction would essentially double the number of possible sets that would need listing out - we also felt that this would make consumer-side behavior potentially more complex as the consumer would have to look through a large number of sets and couldn't assume that those sets were just a long hand version of a few [typically] binary choices.

>Roberta: We can think of providing a description of capture devices in a scene, and then link the captures to their capture device.
>Roberta: In other words, we can  convey information about which captures are mutually exclusive rather than provide possible alternatives of simultaneous captures.

Yes, I think all of the above is basically agreeing with your points, though we thought it useful to not introduce the notion of physical devices and the mapping of the more abstract capture devices onto them, as there are lots of cases where there might not be such physical devices (e.g. an MCU, or playing back from a recording). To my thinking, all of the benefit of this approach is achieved by having the abstract sets of captures which cannot be provided together, with the additional benefit that systems sufficiently simple to have no such restrictions could just omit these sets completely.

The hypothesised MCU case above where 1, 2, 3 and 4 capture versions of the same scene were desired not to be provided together is more problematic with this idea though, and that case would seem to work better in an environment where the sets of captures that *could* be provided together were listed, instead of sets of captures that could not. However, we felt that this was likely to be less common...

Hope this helps,

Andy

On Mon, Dec 3, 2012 at 6:58 PM, Roberta Presta <[email-masked]<mailto:[email-masked]>> wrote:
Hi all,

I have a couple of questions about the use of simultaneous sets.
I have a camera that can be used to capture (i) a zoomed-out view of the telepresence room (say VC1)  and a (ii) moving zoomed-in view of the current speaker (say VC2).
VC1 and VC2 can not clearly be sent simultaneously, then they never appear in the same simultaneous set.

Question nr1: If I listed all the possible simultaneous sets, should I have to provide all the possible combinations of captures without having together VC1 and VC2?

Question nr2: What I understand is that simultaneity constraints that would be expressed by using simultaneous sets are mainly related to the impossibility of sending simultaneously different captures that are captured by the same device.
Is it the only motivation because of which the simultaneous sets have been designed?
Indeed, other constraints related to the simultaneous sending of multiple captures can be deduced using the encodings bandwidth information.

If the answer to Question nr2 is "no", can you please provide other cases in which simultaneous sets are needed (Question nr3)? They are not reported in the framework document.
Otherwise, If the answer to Question nr2 is "yes", maybe there is an easier way to handle the issue.
We can think of providing a description of capture devices in a scene, and then link the captures to their capture device.
It goes straightforward that two captures of the same media type taken from the same capture device can not be sent simultaneously.
In other words, we can  convey information about which captures are mutually exclusive rather than provide possible alternatives of simultaneous captures.

Cheers,
Roberta

_______________________________________________
clue mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/clue

---

From: Christian Groves <[email-masked]>
Date: Fri, 07 Dec 2012 11:28:34 +1100

Hello,

Please see my replies below.

Regards, Christian

On 5/12/2012 4:32 AM, Andy Pepperell wrote:
> - the first is the one you mention, multiple media captures that use 
> the same physical device with, say, different P/T/Z values, which 
> cannot be provided at the same time
> - the second case was for something like a switched or transcoded MCU 
> which might provide 1, 2, 3 and 4 capture alternate views of a 
> conference, but might not want to be obliged to provide, say, the 
> leftmost capture of 2 and the centre capture of 3 simultaneously.
>
> As part of the framework, we thought a little about the best way to 
> express these restrictions; the first case lends itself well to a 
> small number of small simultaneous transmission sets, which each set 
> listing just the captures that cannot be provided together. Our 
> feeling was that listing all of the sets which *could* be provided 
> together could lead to a combinatorial explosion as every pair of 
> captures which had a restriction would essentially double the number 
> of possible sets that would need listing out - we also felt that this 
> would make consumer-side behavior potentially more complex as the 
> consumer would have to look through a large number of sets and 
> couldn't assume that those sets were just a long hand version of a few 
> [typically] binary choices.

[CNG] But doesn't the framework specify which captures CAN be sent 
together? i.e. from sect 6.3 "Simultaneous transmission sets are 
expressed as sets of the MCs that could physically be transmitted at the 
same time, (though it may not make sense to do so).

>
> >Roberta: We can think of providing a description of capture devices 
> in a scene, and then link the captures to their capture device.
> >Roberta: In other words, we can  convey information about which 
> captures are mutually exclusive rather than provide possible 
> alternatives of simultaneous captures.
>
> though we thought it useful to not introduce the notion of physical 
> devices and the mapping of the more abstract capture devices onto 
> them, as there are lots of cases where there might not be such 
> physical devices (e.g. an MCU, or playing back from a recording). To 
> my thinking, all of the benefit of this approach is achieved by having 
> the abstract sets of captures which cannot be provided together, with 
> the additional benefit that systems sufficiently simple to have no 
> such restrictions could just omit these sets completely.
[CNG] I agree I think its better to use an abstract concept.
>
> The hypothesised MCU case above where 1, 2, 3 and 4 capture versions 
> of the same scene were desired not to be provided together is more 
> problematic with this idea though, and that case would seem to work 
> better in an environment where the sets of captures that *could* be 
> provided together were listed, instead of sets of captures that could 
> not. However, we felt that this was likely to be less common...
>
> Hope this helps,
>
> Andy
>
>
> On Mon, Dec 3, 2012 at 6:58 PM, Roberta Presta 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     Hi all,
>
>     I have a camera that can be used to capture (i) a zoomed-out view
>     of the telepresence room (say VC1)  and a (ii) moving zoomed-in
>     view of the current speaker (say VC2).
>     VC1 and VC2 can not clearly be sent simultaneously, then they
>     never appear in the same simultaneous set.
>
>     Question nr1: If I listed all the possible simultaneous sets,
>     should I have to provide all the possible combinations of captures
>     without having together VC1 and VC2?
>
>     Question nr2: What I understand is that simultaneity constraints
>     that would be expressed by using simultaneous sets are mainly
>     related to the impossibility of sending simultaneously different
>     captures that are captured by the same device.
>     If the answer to Question nr2 is "no", can you please provide
>     other cases in which simultaneous sets are needed (Question nr3)?
>     They are not reported in the framework document.
>     We can think of providing a description of capture devices in a
>     It goes straightforward that two captures of the same media type
>     taken from the same capture device can not be sent simultaneously.
>     In other words, we can  convey information about which captures
>     are mutually exclusive rather than provide possible alternatives
>     of simultaneous captures.
>
>     Cheers,
>     Roberta
>
>
>         I agree the framework (or other document, after refactoring)
>         needs to be clear about whether expressing simultaneous
>         transmission sets is mandatory or not.  Maybe it should be
>         mandatory for the media provider to express a simultaneous
>         transmission set that includes all the media captures, even if
>         it has no restrictions.
>
>
>
>     [email-masked] <mailto:[email-masked]>

---

From: Christian Groves <[email-masked]>
Date: Fri, 07 Dec 2012 11:43:10 +1100

Hello Paul,

It would offer flexibility but is this added flexibility needed? In 
order to choose multiple capture scene entries the original provider 
must provide a simultaneous transmission so the MCU knows what it can 
get (we still don't know if its intended that STSs are mandatory or 
optional in an advertisement). If there's multiple parties involved with 
the MCU then the combinations of captures in going to quickly increase I 
don't know if allowing yet more alternatives is beneficial.

Did you have a concrete case in mind?

Regards, Christian

On 4/12/2012 2:35 AM, Paul Kyzivat wrote:
>> Regarding 1), I see very little reasons for allowing it. Of course, 
>> it shall be possible to receive multiple capture scene entries, 
>> associated with *different* scenes. A typical example is one entry 
>> associated with the "room", and one associated with a "presentation".
>
> This would give it the flexibility to offer a wider range of choices 
> in its own advertisements.
>
>     Thanks,
>     Paul
>
>> Regarding 2), during the SDP offer/answer, nothing can prevent the 
>> answerer from rejecting streams associated with specific captures. 
>> But, I am not sure we need to cover it in the CLUE signaling. In the 
>> CLUE signaling the answerer indicates which capture scene entry it 
>> wants - including all captures associated with it.
>
>> Regards,
>>
>> Christer
>>
>>
>> I'm no strongly against being able to select individual captures 
>> however it seems to me to add complications and I think that if we 
>> were to continue to go in that direction could benefit from further 
>> discussion in the framework.
>>
>> What isn't clear to me in the current framework is whether the 
>> sending of simultaneous transmission sets is mandatory and if not 
>> what omitting this part of an advertisement means?
>>
>>> Points a) and b) are consistent with the intent of the current 
>>> framework.  Point d) starts to change the intent.  The framework 
>>> allows for a media consumer to receive multiple capture scene 
>>> entries (regardless of media type) at the same time.  I think point 
>>> d) is proposing we not allow this.  I think saying "A consumer shall 
>>> choose a capture scene entry rather than choosing individual 
>>> captures from multiple entries" is equivalent to saying "A consumer 
>>> shall choose at most one capture scene entry (of a particular media 
>>> type) from any particular capture scene".
>>>
>>> Such a restriction probably reduces the scenarios in which the 
>>> provider needs to express simultaneous transmission sets, because 
>>> some of the simultaneity constraints are now included in the capture 
>>> scene entries.  As others have pointed out, there is still the 
>>> possibility of simultaneous constraints that apply across multiple 
>>> capture scenes.
>>> Adding this restriction, to not allow multiple capture scene entries 
>>> (of same media type, in the same scene) to be used simultaneously is 
>>> okay with me, but I don't think it really simplifies much.
>>>
>>> As others have said, allowing a consumer to choose a subset of 
>>> captures from a capture scene entry makes sense to me too.
>>>
>>> [Duckworth, Mark] These three proposed changes above are related to 
>>> not allowing multiple capture scene entries, of the same media type, 
>>> in the same scene, from being used simultaneously.  I could go 
>>> either way on this change, whatever the group decides.
>>>
>>> these captures (VC0, VC1 and VC2) for rendering purposes is 
>>> accomplished through use of their Area of Capture attributes."  I 
>>> don't know what other type or order or prioritization you might be 
>>> talking about here.
>> rendered the other aspect is regarding how the consumer interprets 
>> the order of scene/capture scene entry/capture within the message. 
>> i.e. Is the first capture scene entry more important than the last 
>> entry? There has been some discussion previously that some people 
>> that the provider of the advertisement could provide some 
>> prioritisation hint to the consumer by ordering the list of capture 
>> scene entries. From the text you mentioned aspect is there a general 
>> conclusion that there is no prioritisation implied from the ordering 
>> of elements in the advertisement?
>>>
>>>> Thoughts?
>>>>
>>>> Regards, Christian

---

From: Christian Groves <[email-masked]>
Date: Fri, 07 Dec 2012 12:21:16 +1100

Hello Mark,

With regards to simultaneous sets I think we need to resolve what 
capture scene entries represent. Currently the framework says that it 
must be possible to send all captures in a Capture Scene Entry 
simultaneously. To me logically speaking a capture scene entry is 
effectively a simultaneous transmission set. To send effectively the 
same information twice seems to me to be abit of a waste.

Related to the above point can different captures types i.e. 
audio/video/presentation be in one simultaneous transmission set ? I.e. 
I can send this video with this audio but not with a presentation. This 
doesn't appear to be addressed in the text.

With regards to prioritisation I'm OK with having the framework not 
imply ordering or prioritisation. After going through the exercise at 
looking at the capture attributes I think that prioritisation maybe 
better as a specific attribute if its needed. I think that if you can 
provide sufficient description of the characteristics of the capture 
then priority may not be needed. Either way I think we agree that it 
needs to be made clear in the framework.

Regards, Christian

---

From: "Duckworth, Mark" <[email-masked]>
Date: Fri, 7 Dec 2012 12:33:06 -0800

Hi Christian,

Responses below.
Mark

> From: Christian Groves [mailto:[email-masked]]
> Sent: Thursday, December 06, 2012 8:21 PM
> To: Duckworth, Mark
> Cc: [email-masked]
> With regards to simultaneous sets I think we need to resolve what capture
> scene entries represent. Currently the framework says that it must be
> possible to send all captures in a Capture Scene Entry simultaneously. To me
> logically speaking a capture scene entry is effectively a simultaneous
> transmission set. To send effectively the same information twice seems to
> me to be abit of a waste.

[Duckworth, Mark] It isn't supposed to be the same information.  The framework says "The simultaneous transmission sets MUST allow all the media captures in a particular capture scene entry to be used simultaneously."  So all the media captures from a particular capture scene entry must also appear together in a simultaneous transmission set.  But that simultaneous transmission set could also include more media captures, not just media captures that appear in the same capture scene entry.

> I can send this video with this audio but not with a presentation. This doesn't
> appear to be addressed in the text.

[Duckworth, Mark] I think the framework intent was that different media types (audio, video, text, ...) have different simultaneous transmission sets.  In other words, the framework doesn't support expressing media provider limitations regarding mutual exclusion between media captures of different media types.   "Presentation" is not a media type in this sense.

> With regards to prioritisation I'm OK with having the framework not imply
> ordering or prioritisation. After going through the exercise at looking at the
> capture attributes I think that prioritisation maybe better as a specific
> attribute if its needed. I think that if you can provide sufficient description of
> the characteristics of the capture then priority may not be needed. Either
> way I think we agree that it needs to be made clear in the framework.

[Duckworth, Mark] Sounds good to me.  It wouldn't hurt to add text that says the order capture scenes and the order of capture scene entries has no significance, either in the framework or a protocol document, depending on how we refactor it.

> simultaneous transmission set that includes all the media captures, even if it
> has no restrictions.
> >
> > About the order of capture scenes, entries within a scene, and captures
> within an entry - the framework intent was that this order had no
> significance.  But it does seem it might be useful to use the ordering as a
> prioritization hint from provider to consumer.  If we decide we want to do
> this, we need to make it clear in the framework (or possibly lower level
> document, if that is where it ends up after refactoring).
> >
> > Mark
> >
> >> Point d) starts to change the intent.  The framework allows for a
> >> media consumer to receive multiple capture scene entries (regardless
> >> of media
> >> type) at the same time.  I think point d) is proposing we not allow
> >> this.  I think saying "A consumer shall choose a capture scene entry
> >> rather than choosing individual captures from multiple entries" is
> >> equivalent to saying "A consumer shall choose at most one capture
> >> scene entry (of a particular media type) from any particular capture
> scene".
> >>> provider
> >> simultaneity constraints are now included in the capture scene
> >> entries.  As others have pointed out, there is still the possibility
> >> of simultaneous constraints that apply across multiple capture scenes.
> >>> (of same
> >> media type, in the same scene) to be used simultaneously is okay with
> >> me, but I don't think it really simplifies much.
> >>> captures
> >>>> I think the framework draft could be enhanced in a few places to
> >>>> make the intent of CLUE clearer. Below are some suggested updates.
> >>>>
> >>>> explanation. I propose the following definition to be added to
> >>>> section 3 of
> >> the draft:
> >>>> 2) Update the definition of "Capture scene entry" to make it
> >>>> clearer that it represents an entire scene. Proposed text is:
> >>>>       the media consumer choose which captures it wants.  The
> >>>> capture
> >> scene
> >>>> - section 6.2 New paragraph dealing with consumer behaviour after
> >>>> the 7th paragraph.
> >>>> choose an capture scene entry in the knowledge that it is a
> >>>> complete representation of the scene for a particular media type
> >>>> and that all media captures are spatially related. For a particular
> >>>> media type the consumer shall choose one capture scene entry rather
> >>>> than choosing individual captures from multiple capture scene
> >>>> entries. However this does not mean that a consuming endpoint must
> >>>> render all the media captures. What is locally rendered and how is a
> local decision."
> >>> not
> >> allowing multiple capture scene entries, of the same media type, in
> >> the same scene, from being used simultaneously.  I could go either
> >> way on this change, whatever the group decides.
> >>>> support is the ordering in the Capture scene, Capture scene entry,
> >>>> Media
> >> captures.
> >>>> prioritisation / ordering related to these structures. i.e. If I
> >>>> have a capture scene entry VC0, VC1, VC2 do I assume they should be
> >>>> is the later understanding put I didn't see anything in the
> >>>> framework regarding what should be assumed. We probably should
> also
> >>>> add
> >> something for that.
> >> don't know what other type or order or prioritization you might be talking
> about here.
> >> conclusion that there is no prioritisation implied from the ordering of
> elements in the advertisement?
> >>>> Thoughts?
> >>>>
> >>>> Regards, Christian

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 10 Dec 2012 08:32:45 +0000

Hi,

> [Duckworth, Mark] It isn't supposed to be the same information.  The framework says "The simultaneous transmission sets MUST allow all the media captures in a 
> particular capture scene entry to be used simultaneously."  So all the media captures from a particular capture scene entry must also appear together in a simultaneous 
> transmission set.  But that simultaneous transmission set could also include more media captures, not just media captures that appear in the same capture scene entry.

I agree to the understanding, but I still question the need.

Why can't the "additional captures" be part of a dedicated capture scene entry?

Regards,

Christer

---

From: Christian Groves <[email-masked]>
Date: Mon, 10 Dec 2012 22:11:49 +1100

Hello Mark,

Please see my response below.

Regards, Christian

> [Duckworth, Mark] It isn't supposed to be the same information.  The framework says "The simultaneous transmission sets MUST allow all the media captures in a particular capture scene entry to be u…

[CNG] I know its not meant to be the same but it does seem like a large 
overlap. Particular if the STS doesn't contain additional media 
captures. This would be likely if the receiver could only choose one 
entry from a list of capture scene entries.

> [Duckworth, Mark] I think the framework intent was that different media types (audio, video, text, ...) have different simultaneous transmission sets.  In other words, the framework doesn't support…
[CNG] If that is the intent then it should be explicitly captured in the 
framework text.
With regards to the "presentation" not being a media type. I guess this 
means that when the Content attribute indicates "Slides" this is only 
valid when the slides are a video stream? i.e. CLUE has no mechanism to 
indicate a data stream related to a presentation.
> [Duckworth, Mark] Sounds good to me.  It wouldn't hurt to add text that says the order capture scenes and the order of capture scene entries has no significance, either in the framework or a protoc…
[CNG] Agreed.

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 10 Dec 2012 13:07:59 +0000

Hi,

>> [Duckworth, Mark] It isn't supposed to be the same information.  The 
>> framework says "The simultaneous transmission sets MUST allow all the 
>> media captures in a particular capture scene entry to be used simultaneously."  So all the media captures from a particular capture scene 
>> entry must also appear together in a simultaneous transmission set.  But that simultaneous transmission set could also include more media captures, not just media captures that appear in the same capture scene entry.
>

...and, if the additional captures can not be part of the capture scene entry, because they don't have spatial relationships with the other captures within the capture scene entry, why not advertise the additional captures as separate scenes?

Regards,

Christer

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 10 Dec 2012 10:01:55 -0500

> Why can't the "additional captures" be part of a dedicated capture scene entry?

I guess the question is whether you think the provider can anticipate 
and advertise every possible combination that a receiver might desire.

An example that has been given various times is a recipient that wants 
to always receive the capture that includes his boss, even when the boss 
isn't speaking. And he wants this in combination with some other 
captures that might be some conventional scene entry. So this might mean 
one full scene entry plus one capture from some other scene entry. Or it 
might mean *parts* of two scene entries to assemble something useful 
that always includes the boss.

(Note: I don't have a strong opinion here. I'm replaying what I've heard.)

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 10 Dec 2012 10:24:35 -0500

> Did you have a concrete case in mind?

I think I forgot to respond to this.
I'm leaving part of the earlier conversation for context, and then I'll 
give one example.

>> in its own advertisements.

Consider a room (room-1) that has seating for six, and six cameras. 
Among the capture scenes it advertises, it might include one entry with 
two captures each including three people. And another entry with three 
captures each including two people. And it can provide these 
simultaneously. (It can't send six individual captures.)

It then connects to an MCU for a conference with two other rooms. Room-2 
has one particpant, one camera, and three screens. Room-3 has one 
participant, one camera, and four screens.

Room-2 could see both other rooms if the MCU would advertise a scene 
entry with two captures from Room-1 and the one capture from Room-3. 
Room-3 would be happier if the MCU would advertise a scene entry with 
three captures from Room-1 and the one capture from Room-2.

(Admittedly this is a bit artificial. There may be some more plausible 
cases.)

	Thanks,
	Paul

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 10 Dec 2012 16:48:46 +0000

Hi,

> isn't speaking.
> And he wants this in combination with some other captures that might be some conventional scene entry. So this might mean
> one full scene entry plus one capture from some other scene entry. Or it might mean *parts* of two scene entries to assemble something useful

You can achieve this e.g. using two separate SCENES: one SCENE with a capture scene entry that contains the full scene capture, and another SCENE with a capture scene entry that contains the boss capture.

...or, if possible (due to spatial relationship etc), you can put both the full scene captures and the boss capture in a single capture scene entry.

For any given SCENE, I think a consumer shall only be allowed to select a single capture scene entry. The consumer may, though, choose multiple SCENES (but, again, only a single capture scene entry from any given SCENE).

(NOTE: I use captial letters for SCENE, to clearly separate between scene and capture scene entry :)

>(Note: I don't have a strong opinion here. I'm replaying what I've heard.)

I just think we should start by trying to keep things simple.

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 10 Dec 2012 17:07:33 +0000

Hi,

> three captures from Room-1 and the one capture from Room-2.

The MCU can advertise different scene entries (or, different scenes) to Room-2 and Room-3.

Regards,

Christer

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 10 Dec 2012 12:40:30 -0500

> You can achieve this e.g. using two separate SCENES: one SCENE with a capture scene entry that contains the full scene capture, and another SCENE with a capture scene entry that contains the boss c…

My assumption was that the boss is not everyone's boss, or known to be 
special in any way to the providing site. Rather he is important only to 
the recipient(s) in one of the other rooms.

Stated more abstractly, the receiver has special interest in one of the 
provider's captures that the provider does not realize is of more 
interest than any of its other captures.

> For any given SCENE, I think a consumer shall only be allowed to select a single capture scene entry.

I'm fine with this simplification if we have consensus that it is 
sufficient.

> The consumer may, though, choose multiple SCENES (but, again, only a single capture scene entry from any given SCENE).

I'm concerned about your use of "may" here, and what that implies about 
the significance of scenes.

While we can't and shouldn't force the consumer to request more than it 
wants, IMO it had better be clear to the consumer what it needs to do to 
get a "complete" or "sufficient" rendition of the call. Creating extra 
scenes for the purpose of advertising "optional/extra" stuff (such as 
the extra scene for the boss) is very different from creating extra 
scenes for stuff that is equally important but spatially unrelated.

> I just think we should start by trying to keep things simple.

As simple as it can be while satisfying all the requirements,
and no simpler. :-)

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 10 Dec 2012 12:48:21 -0500

> The MCU can advertise different scene entries (or, different scenes) to Room-2 and Room-3.

Certainly it can. If fact it typically will, since it probably won't 
want the advertisement to Room-N to include captures received from Room-N.

But I don't think that has any bearing on this discussion, for a couple 
of reasons:

- To customize the advertisements to include different things from
   Room-1, it would need to know something about what Room-2 and Room-3
   want or can use. But the point is to advertise what is possible and
   let the recipient decide what it wants to receive.

- In any case, if the result is that Room-2 can request two 3-person
   captures of Room-1 and Room-3 can request three 2-person captures
   of Room-1, then the MCU will need to request all of those from
   Room-1.

	Thanks,
	Paul

---

From: John Leslie <[email-masked]>
Date: Mon, 10 Dec 2012 14:55:46 -0500

Christer Holmberg <[email-masked]> wrote:
> 
> For any given SCENE, I think a consumer shall only be allowed to select
> a single capture scene entry. The consumer may, though, choose multiple
> SCENES (but, again, only a single capture scene entry from any given
> SCENE).

   Not meaning to speak for or against this, it seems a bit confusing to
impose this limitation on a person.

   Could you clarify what benefit this restriction brings us?

--
John Leslie <[email-masked]>

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 12 Dec 2012 11:27:06 +0000

Hi,

>> For any given SCENE, I think a consumer shall only be allowed to 
>> select a single capture scene entry. The consumer may, though, choose 
>> multiple SCENES (but, again, only a single capture scene entry from 
>> any given SCENE).
>
>   Not meaning to speak for or against this, it seems a bit confusing to impose this limitation on a person.
>

Simplicity.

Capture scene entries provide alternatives for a given SCENE, and I think the receiver will chose one alternative.

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 12 Dec 2012 11:40:27 +0000

Hi,

>>>>>> capture scene entries represent. Currently the framework says that 
>>>>>> it must be possible to send all captures in a Capture Scene Entry 
>>>>>> effectively a simultaneous transmission set. To send effectively 
>>>>>> the same information twice seems to me to be abit of a waste.
>>>>>
>>>>> [Duckworth, Mark] It isn't supposed to be the same information.  
>>>>> The framework says "The simultaneous transmission sets MUST allow 
>>>>> all the media captures in a particular capture scene entry to be used simultaneously."  So all the media captures from a particular capture 
>>>>> scene entry must also appear together in a simultaneous transmission set.  But that >>>>>simultaneous transmission set could also include more media captures, not just media captures that appear in the same capture scene entry.
>>>>
>>> An example that has been given various times is a recipient that 
>>> wants to always receive the capture that includes his boss, even when 
>>> the boss isn't speaking.
>>>
>>> And he wants this in combination with some other captures that might 
>>> be some conventional scene entry. So this might mean one full scene 
>>> entry plus one capture from some other scene entry. Or it might mean *parts* of two scene entries to assemble something useful that always includes the boss.
>>
> My assumption was that the boss is not everyone's boss, or known to be special in any way to the providing site. Rather he is important only to the recipient(s) in one of the other rooms.
>
> Stated more abstractly, the receiver has special interest in one of the provider's captures that the provider does not realize is of more interest than any of its other captures.

So, if I understand you correctly, the provider could e.g:

1) Provide a single SCENE with a capture scene entry that contains the "full room", and individual captures of each participant. Of the individual participant captures, the receiver will then only choose the capture associated with the important participant - ie the receiver is not mandated to receive everything in the chosen capture scene entry;

2) Provide a SCENE with a capture scene entry that contains the "full room", and in addition individual SCENEs for each individual participant. The receiver will then choose the "full room" SCENE, and the SCENE representing the important participant;

3) Similar to 1), but provide a single SCENE with multiple capture scene entries, where each entry contains the "full room", and the capture of ONE participant. The receiver will then choose the capture scene entry which contains the capture associated with the important participant. 

Etc.

I am not claiming that the 3 alternatives above are equally smart, but my point is that I think we can solve use-cases by using SCENEs and capture scene entries.

> I'm concerned about your use of "may" here, and what that implies about the significance of scenes.
>
> While we can't and shouldn't force the consumer to request more than it wants, IMO it had better be clear to the consumer what it needs to do to get a "complete" or "sufficient" rendition of the call. Creating extra scenes for the 
> purpose of advertising "optional/extra" stuff (such as the extra scene for the boss) is very different from creating extra scenes for stuff that is equally important but spatially unrelated.

Fair enough - then, assuming the boss capture is considered spatially related to the "full room", choose alternative 1) above :)

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 12 Dec 2012 11:50:01 +0000

Hi,

>>>> must provide a simultaneous transmission so the MCU knows what it 
>>>> can get (we still don't know if its intended that STSs are mandatory 
>>>> or optional in an advertisement). If there's multiple parties 
>>>> involved with the MCU then the combinations of captures in going to 
>>>> quickly increase I don't know if allowing yet more alternatives is beneficial.
>>>>
>>> I'm leaving part of the earlier conversation for context, and then 
>>> I'll give one example.
>>>
>>> Among the capture scenes it advertises, it might include one entry 
>>> with two captures each including three people. And another entry with 
>>> three captures each including two people. And it can provide these 
>>> It then connects to an MCU for a conference with two other rooms. 
>>> Room-2 has one particpant, one camera, and three screens. Room-3 has 
>>> one participant, one camera, and four screens.
>>>
> Certainly it can. If fact it typically will, since it probably won't want the advertisement to Room-N to include captures received from Room-N.

Yes.

> But I don't think that has any bearing on this discussion, for a couple of reasons:
>

I think Room-1 should advertise what it can do, and Room-2 and Room-3 then chooses.

Again, if we assume that Room-2 and Room-3 can choose not to receive certain parts of a capture scene entry, I am pretty sure Room-2 and Room-3 will find something that fits.

>   of Room-1, then the MCU will need to request all of those from

I doubt whether Room-1 will be able to provide both 3-person captures and 2-person captures at the same time.

But, assuming it can, why can't they then be part of the same capture scene entry provided by Room-1, and the MCU then tells Room-1 what it wants from that capture scene entry?

I think it's much better to allow a receiver to indicate that it does NOT want certain parts from a given capture scene entry, rather than trying to compose a new capture scene entry by picking stuff from different capture scene entries...

Regards,

Christer

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 12 Dec 2012 12:19:33 -0500

More inline

> So, if I understand you correctly, the provider could e.g:

There are many things a provider *could* do. Many of them don't make 
sense, or IMO are a bad idea.

> 1) Provide a single SCENE with a capture scene entry that contains the "full room", and individual captures of each participant.

I guess you mean that a single scene *entry* contains both the "full 
room" capture and the individual captures?

IMO that is contrary to the intent of a scene entry. The entry has 
redundant information, and requires a recipient with no special needs to 
analyze it and decide it could simply render *either* the "full room" 
capture, or all the other captures in the entry. This is hard.

Rather, I would think this same information would be represented better 
as two scene entries:

- an entry with the single "full room" capture
- an entry with individual captures for each participant.

Each of these provides a full rendering of the room, in a different way.

> Of the individual participant captures, the receiver will then only choose the capture associated with the important participant - ie the receiver is not mandated to receive everything in the chosen capture scene entry;

With the alternative I just mentioned, a receiver with no special 
interests could then choose one of those entries that best meets its 
needs and capabilities. E.g. if it has enough displays and decoders to 
receive all the individual captures, then that might be the better 
choice. If it doesn't have enough resources to do that then it could 
simply choose the single capture full room entry.

A receiver that has special interest in one participant (the boss) might 
then decide to receive the entry with the full room capture, and also 
the one capture for the boss from the entry that has all the individual 
captures.

> 2) Provide a SCENE with a capture scene entry that contains the "full room", and in addition individual SCENEs for each individual participant. The receiver will then choose the "full room" SCENE,…

This doesn't make sense to be. All of those captures share the same 
coordinate space, so they ought to be in the same scene.

> 3) Similar to 1), but provide a single SCENE with multiple capture scene entries, where each entry contains the "full room", and the capture of ONE participant. The receiver will then choose the ca…

This also doesn't make much sense to me. It has the same problems as (1).

> I am not claiming that the 3 alternatives above are equally smart, but my point is that I think we can solve use-cases by using SCENEs and capture scene entries.

This has just moved the complexity. It makes it harder for recipients 
that have no special needs and just want to find an entry to render that 
fits their capabilities. And it makes it harder for the provider to 
anticipate combinations that are only of rare and special interest.

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 12 Dec 2012 12:30:58 -0500

> I think Room-1 should advertise what it can do, and Room-2 and Room-3 then chooses.

The problem is that Room-1 advertises what it can do to the *MCU*, not 
to Room-2 and Room-3. The MCU then must decide what to advertise to 
Room-2 and Room-3.

In the case I described, it happens that it is possible to let everybody 
see everybody else simultaneously without doing any switching or 
composing in the MCU. So what I described has the MCU directly "passing 
through" the captures it receives. In reality it would probably *also* 
advertise some other entries that are switched or composed in order to 
accommodate receivers that can only handle fewer (e.g. one) captures.

> Again, if we assume that Room-2 and Room-3 can choose not to receive certain parts of a capture scene entry, I am pretty sure Room-2 and Room-3 will find something that fits.

My point was that with the MCU passing everything through, both Room-2 
and Room-3 *can* find something that works for them. That then results 

> I doubt whether Room-1 will be able to provide both 3-person captures and 2-person captures at the same time.

I described the scenario such that it could. Perhaps this is 
unrealistic. If so then it would have indicated that by not including 
them in the simultaneous sets.

> But, assuming it can, why can't they then be part of the same capture scene entry provided by Room-1, and the MCU then tells Room-1 what it wants from that capture scene entry?

I explained this in my reply to your prior message. Including them in 
the same scene entry fills that entry with redundant information and 
makes it difficult for most receivers to use.

> I think it's much better to allow a receiver to indicate that it does NOT want certain parts from a given capture scene entry, rather than trying to compose a new capture scene entry by picking stu…

This is just moving complexity around, not reducing it. (And perhaps 
increasing it.)

	Thanks,
	Paul

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 12 Dec 2012 20:37:27 +0000

Hi,

I'm putting the other discussion on hold for the moment, because I think we are starting to find some common ground in this one :)

> There are many things a provider *could* do. Many of them don't make sense, or IMO are a bad idea.
>
> I guess you mean that a single scene *entry* contains both the "full room" capture and the individual captures?
>

Ok, let's go with that for now :)

So, we have a single SCENE, which has two associated CSEs (Capture Scene Entries):

- "full room" CSE (capture of full room)
- "individual" CSE (captures of each participant)

> simply choose the single capture full room entry.

Agree. 

> the one capture for the boss from the entry that has all the individual

Almost agree :)

In my opinion, the receiver would decide two receive both CSEs, but it would indicate that from the individual CSE it only wants to "boss" capture.

So, the end result is really the same. The only different is that, in my opinion, the receiver must always pick a CSE (but it may choose which captures it wants).

The advantage is that we then only need to advertise simultanous CSE sets - not simultanous capture sets - as it is assumed that a provider can always provide all captures associated with a CSE.

(This of course means we would need to allow picking multiple CSEs from a single SCENE.)

Regards,

Christer

---

From: John Leslie <[email-masked]>
Date: Wed, 12 Dec 2012 17:39:59 -0500

Christer Holmberg <[email-masked]> wrote:
> [John Leslie <[email-masked]> wrote:]
>> [Christer Holmberg <[email-masked]> wrote:]
> 

   (I wonder if we're all reading this the same way...)

>> impose this limitation on a person.

   (by which I mean, I'd prefer that software not impose such a limit
on participants...)

>> Could you clarify what benefit this restriction brings us?

   While simplicity is an admirable goal, it's not obvious to me that
the restriction simplifies anything important. (YMMV, of course...)

> Capture scene entries provide alternatives for a given SCENE, and
> I think the receiver will chose one alternative.

   Indeed, most users might always choose one capture scene entry.
But I have experienced _many_ users over the years that viscerally
resent being told to choose "only one".

   I advise against designing capabilites based on what _we_ think
"most users" will want.

---

From: Christian Groves <[email-masked]>
Date: Thu, 13 Dec 2012 11:56:09 +1100

Hello Paul,

Please see my response below:
..snip..
> In the case I described, it happens that it is possible to let 
> everybody see everybody else simultaneously without doing any 
> switching or composing in the MCU. So what I described has the MCU 
> directly "passing through" the captures it receives. In reality it 
> would probably *also* advertise some other entries that are switched 
> or composed in order to accommodate receivers that can only handle 
> fewer (e.g. one) captures.

[CNG] This is the point I was trying to understand, how you saw the MCU 
being involved. You see the MCU as simply passing through the CLUE 
Advertisements. In my mind I see CLUE not as an end-to-end protocol. 
I.e. the MCU would effectively act as a CLUE endpoint and 
"re-configures" any advertisements/configures sent to other endpoints.
I think this aspect is worth examining further. Do you in CLUE allow a 
situation for a MCU to effectively "broadcast" advertisements i.e. from 
room 1 to room 2 & 3 and then allow these rooms to response directly to 
Room 1? etc.

---

From: Christian Groves <[email-masked]>
Date: Thu, 13 Dec 2012 12:06:12 +1100

Hello,

I think we need to distinguish that the person sitting in the conference 
is not the immediate user of CLUE. I wouldn't expect a human to parse 
the CLUE advertisement. No doubt there will be some GUI presenting lots 
of different solutions.

Regards, Christian

---

From: Christian Groves <[email-masked]>
Date: Thu, 13 Dec 2012 12:53:20 +1100

Hello

I'm not sure about this. Now we are saying that a capture scene entry 
does not represent an "entire scene" but now represents part of a scene. 
I think logically it works better if an individual capture is part of 
the scene and a capture scene entry depicts an "entire" scene from the 
viewpoint of the provider. If the provider now provides part scenes how 
does the consumer determine what's a full scene?

It seems to me that whilst a Consumer may theoretically want to do all 
sorts of things with the captures ultimately its the provider that 
determines what can actually be provided. The consumer may just want to 
see "the boss" but the provider won't know that it has to provide 
individual captures when it does the initial advertisement.

A provider may be reasonably concise (i.e. I have two capture scene 
entries choose one) or it may be verbose (i.e. I have several capture 
scene entries, I can provide all these different captures, here's the 
dependency between them).

I've asked previously if simultaneous transmission sets are a mandatory 
to send. I don't think this has been determined.

After the discussions in order to strike a balance between simple and 
complex cases i'd like to propose the following:

1. It is optional for a provider to send a simultaneous transmission set.
2. If the provider does not send a simultaneous transmission set:
2a. if multiple scenes are advertised it is assumed that they can be 
provided simultaneously.
2b. if multiple capture scene entries are sent in a scene then the 
consumer must chose one capture scene entry.
3. If the provider does sent a simultaneous transmission set:
3a. if multiple scenes are advertised simultaneity constraints are 
derived from the simultaneous transmission set.
3b. if multiple capture scene entries are sent in a scene then the 
consumer SHOULD close on capture scene entry but may chose captures 
based on the simultaneous transmission set.

Regards, Christian

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 13 Dec 2012 09:49:13 +0000

Hi,

>I'm not sure about this. Now we are saying that a capture scene entry does not represent an "entire scene" but now represents part of a scene. 
>
> I think logically it works better if an individual capture is part of the scene and a capture scene entry depicts an "entire" scene from the viewpoint of the provider. If the provider now provides part scenes how does the consumer 
> determine what's a full scene?

I agree, and my understanding that in the example both CSEs DO represent an entire scene: one CSE contains a single "full room" capture, and the other CSE contains individual participant captures (all participant captures together forms the entire scene).

> It seems to me that whilst a Consumer may theoretically want to do all sorts of things with the captures ultimately its the provider that determines what can actually be provided. The consumer may just want to see "the boss" but the 
> provider won't know that it has to provide individual captures when it does the initial advertisement.

But, if the provider CAN provide individual captures, I think it would advertise them. If it can't, then the receiver cannot request the "boss" capture.

> A provider may be reasonably concise (i.e. I have two capture scene entries choose one) or it may be verbose (i.e. I have several capture scene entries, I can provide all these different captures, here's the dependency between them).

Exactly. And, in the example below, the provider would indicate that it can provide both CSEs simultaneously. 

> I've asked previously if simultaneous transmission sets are a mandatory to send. I don't think this has been determined.

In this case we would be talking about simultaneous capture scene entries.

>After the discussions in order to strike a balance between simple and complex cases i'd like to propose the following:
>
>2a. if multiple scenes are advertised it is assumed that they can be provided simultaneously.

I think that is ok.

>2b. if multiple capture scene entries are sent in a scene then the consumer must chose one capture scene entry.

Well, in order for the example to work, the receiver would have to be able to choose both the "full scene" CSE and the "individual" CSE (from the "individual" CSE the receiver would then only take the boss capture".

But, again, the provider would have to explicitly indicate that it can provide both CSEs simultaneously.

>3a. if multiple scenes are advertised simultaneity constraints are derived from the simultaneous transmission set.
>3b. if multiple capture scene entries are sent in a scene then the consumer SHOULD close on capture scene entry but may chose captures based on the simultaneous transmission set.

If the receiver is allowed to choose captures from multiple CSEs (associated with the same scene), why can't it then choose multiple CSEs?

Regards,

Christer

On 13/12/2012 7:37 AM, Christer Holmberg wrote:
> Hi,
>
> I'm putting the other discussion on hold for the moment, because I 
> think we are starting to find some common ground in this one :)
>
>>>>>>>>> With regards to simultaneous sets I think we need to resolve 
>>>>>>>>> what capture scene entries represent. Currently the framework 
>>>>>>>>> says that it must be possible to send all captures in a 
>>>>>>>>> Capture Scene Entry simultaneously. To me logically speaking a 
>>>>>>>>> capture scene entry is effectively a simultaneous transmission 
>>>>>>>>> set. To send effectively the same information twice seems to me to be abit of a waste.
>>>>>>>> The framework says "The simultaneous transmission sets MUST 
>>>>>>>> allow all the media captures in a particular capture scene 
>>>>>>>> entry to be used simultaneously."  So all the media captures from a particular capture scene entry must also appear together in a simultaneous transmission set.  But that >>>>>simultaneous transmission set could also include more media captures, not just media captures that appear in the same capture scene entry.
>>>>>> I guess the question is whether you think the provider can 
>>>>>> anticipate and advertise every possible combination that a receiver might desire.
>>>>>>
>>>>>> wants to always receive the capture that includes his boss, even 
>>>>>> when the boss isn't speaking.
>>>>>>
>>>>>> And he wants this in combination with some other captures that 
>>>>>> might be some conventional scene entry. So this might mean one 
>>>>>> full scene entry plus one capture from some other scene entry. Or it might mean *parts* of two scene entries to assemble something useful that always includes the boss.
>> redundant information, and requires a recipient with no special needs 
>> to analyze it and decide it could simply render *either* the "full room"
>> Rather, I would think this same information would be represented 
>> better as two scene entries:
>>
>>> Of the individual participant captures, the receiver will then only 
>>> choose the capture associated with the important participant - ie 
>>> the receiver is not mandated to receive everything in the chosen 
>>> capture scene entry;
>> needs and capabilities. E.g. if it has enough displays and decoders 
>> to receive all the individual captures, then that might be the better 
>> A receiver that has special interest in one participant (the boss) 
>> might then decide to receive the entry with the full room capture, 
>> and also the one capture for the boss from the entry that has all the 
>> individual captures.
> Almost agree :)
>

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 13 Dec 2012 16:24:16 +0000

Hi all,

On 12.12.2012 16:56 , "Christian Groves" <[email-masked]>
wrote:

>[...]

>room 1 to room 2 & 3 and then allow these rooms to response directly to

I don't want to see this important architectural point to be lost in the
Capture Scene thread.

Obviously, an MCU needs to intercept and (in many cases) interpret many of
the CLUE messages. 

The first question is: does the MCU also manipulate and/or potentially
create on its own, CLUE messages.  I believe the answer is "yes".  We need
this, because only the MCU knows whether, and how, it tiles up pictures in
a video mixing scenario (there are many more use cases like this).

The second question is: does the CLUE protocol suite "allow" an MCU to
"effectively broadcast advertisements" (as Christian has put it), without
manipulating them?  My gut reaction to this is" let's not overshoot in
terms of flexibility.  I think we should make it clear that it is the
responsibility of an MCU to consolidate room information it has received
from receiving rooms and provide, based on its unified view, sensible
choices for things like the selection of Capture scenes.

(This is a bit of a departure from what I suggested in one of the early
Andover meetings, where I suggested that everyone in the conference should
have a full understanding of what's going on, so to be able to make
sensible choices.  The issue with this approach is that every vendor will
have a different view what a "sensible" choice may be, and the result
could well be a reduced user experience--and no one to blame for it.  If
the control lives in the MCU, then the MCU vendor is to blame, and no one
else).

Stephan

---

From: John Leslie <[email-masked]>
Date: Thu, 13 Dec 2012 15:43:16 -0500

> a video mixing scenario (there are many more use cases like this).

   I believe an MCU needs to create its own CLUE messages when it pieces
together something not produced (in the same form) at an endpoint.

   I recommend a lot of leeway in what an MCU may stamp as its own.

   I'm less clear on whether a MCU should ever relay CLUE messages,
with or without modification. I'm outright uncomfortable with a MCU
pretending to be the producer instead of an aggregator.

> choices for things like the selection of Capture scenes.

   "Broadcast" sounds scary. It's one thing to advertise its aggregated
scenes/whatever, but I'd be very hesitant to "broadcast" something to
be provided by a different node. (I envision cases where the connectivity
changes so that the path from receiver to source no longer passes through
the same MCU.)

> the control lives in the MCU, then the MCU vendor is to blame, and no one

   I'm not sure that placing blame "exactly" is all that helpful.

   (I also suspect that _intentional_ configuration will lead to "reduced
user experienct" regardless of what we specify.)

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 13 Dec 2012 16:14:31 -0500

The "Capture scene clarifications" topic has been *very* popular, and is 
getting confusing.

Here I've changed the subject line to focus on one point - a change that 
was been proposed by Christer. The latest description of it was from 

>> render all the media captures. What is locally rendered and how is a local

There have been some suggestions that this could also eliminate the need 
for simultaneous sets, because the scene entries would *be* the 
simultaneous sets. I'm not sure if that is an integral part of this 
proposal or not. (If it is, then it provides part of the justification. 
Without this part I'm not sure what the benefit is.)

I am inclined to agree that a new definition of advertisement and 
configuration could be built around such a concept. But to do so I think 
somebody will need to work out all the details. For instance:
- can the same capture be in more than one capture scene entry?
- how does the configure message specify which entries it wants,
   and which captures from those entries?

IMO the existing (prior?) model was at least clearer, and arguably 
simpler. The way I have been thinking of it:

- an advertisement advertises a bunch of captures, and information
   about those captures to help the receiver decide which ones it
   wants and how it should render them.

- scenes, scene entries, simultaneous sets, are all just pieces of
   that descriptive info.

- a configuration is simple - it only needs to enumerate the
   captures that it wants. It need not mention scenes or scene
   entries. (It might need to identify scenes if the capture ids
   are scoped to a scene.)

The proposed changes would make the configuration message more complex. 
The receiver would need to enumerate the desired scene entries. And for 
each selected scene entry, identify those captures that are (or are not) 
desired.

So, IMO, if some people are interested in this new approach, I suggest 
they work on a detailed proposal for how it would work, and then do a 
full assessment of the pros/cons relative to the existing proposal.

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 13 Dec 2012 17:00:41 -0500

>> room 1 to room 2 & 3 and then allow these rooms to response directly to

In the particular case I constructed here the MCU is, more or less, 
"passing through" the advertisements. I did not intend that to be viewed 
as the "normal" or "only" behavior of an MCU. I didn't even mean to 
imply that the MCU in this case included *only* passed through captures. 
I was trying to construct a case to make a point. In practice there 
would probably be other entries that had switched and/or composed captures.

When an MCU has only two attached endpoints I would expect that it would 
be quite likely to pass through captures, and perhaps whole scenes. When 
you step up to three endpoints its unlikely that whole scenes will be 
passed through, but it may make sense to expose all the captures. The 
more endpoints there are, the more important switched and/or composed 
captures are, and the less likely that it makes sense to pass through 
individual captures.

> a video mixing scenario (there are many more use cases like this).

Yes, I agree.

> choices for things like the selection of Capture scenes.

Yes. IMO it should be *valid* for a MCU to "pass through" whole scenes, 
or selected captures from scenes, of the endpoints, or do switching or 
composition, or any combination of those things.

> the control lives in the MCU, then the MCU vendor is to blame, and no one

Agreed.

	Thanks,
	paul

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 13 Dec 2012 17:23:52 -0500

Yes.

>>> Of the individual participant captures, the receiver will then only choose the capture associated with the important participant - ie the receiver is not mandated to receive everything in the cho…

A receiver can choose what it wants.

If it has enough screens, it might choose all the captures from the 
"individual CSE" and display them on separate screens.

Or it might choose all of those captures and compose them itself to fit 
on some lesser number of screens, in an arrangement that suits it.

Or, in the case at hand, it might chose the "full room" CSE, and in 
addition choose one of the individual captures that is of special 
interest to it. The advertiser doesn't need to anticipate that the 
receiver might want to do this. It probably didn't construct its 
advertisement specifically for this case. It is more a matter that the 
receiver can exploit the advertisement to accomplish a goal 
unanticipated by the advertiser.

> In my opinion, the receiver would decide two receive both CSEs, but it would indicate that from the individual CSE it only wants to "boss" capture.

We should try to separate considerations of what is possible from how 
advertisements and configurations are represented. And from how the 
choices might be exposed to end users. (I doubt that an end user will be 
examining the XML representation of the advertisements.)

> (This of course means we would need to allow picking multiple CSEs from a single SCENE.)

I started a subthread with a separate subject line to talk about this.
Can we move the discussion of that there?

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 13 Dec 2012 18:05:26 -0500

> pretending to be the producer instead of an aggregator.

I can see how it *might* be reasonable to do this when the MCU has only 
two connected endpoints. In this case it can in effect act as a proxy 
for CLUE messages.

But I suspect when we get all the details worked out it won't be that 
simple. It might be able to pass through an advertisement that is very 
similar to the one it receives, but probably not identical. Partly that 
will be due to coupling between the CLUE messages and the SDP.

> changes so that the path from receiver to source no longer passes through

I doubt "broadcast" is a suitable word for any valid behavior.

The closest that I can see to this is for the MCU to send out 
advertisements that includes a number of scenes, each corresponding to 
one of the scenes it received from the "other" endpoints.

As noted above, that seems plausible for two endpoints. It gets 
progressively less plausible for increasing numbers of endpoints.

	Thanks,
	Paul

---

From: Andy Pepperell <[email-masked]>
Date: Fri, 14 Dec 2012 17:19:06 +0000

>>For a particular media type the consumer shall choose one capture
>>scene entries
for simultaneous sets,
>because the scene entries would *be* the simultaneous sets.

While I can see the appeal of this approach, I think we shouldn't discount
cases where a consumer might want to configure media captures from more
than one capture scene entry within a scene. An MCU, for instance, might
want to receive a multi-camera view from a room *and* a single roving
camera in order to switch both out to different participants depending on
those participants' rendering capabilities. Similarly, a recording device
might choose to receive from an MCU both a set of switched loudest speaker
captures and a more simple pre-composed view to allow later playback by
devices whose capabilities similarly differ.

(also, we planned for capture scene entries to be restricted to a single
media type (e.g. audio or video) and so it should perhaps be clarified
whether the "shall choose one capture scene entry" should more precisely be
"shall choose one capture scene entry of each media type").

>PaulK: IMO the existing (prior?) model was at least clearer, and arguably
simpler.

I agree with Paul here! (perhaps unsurprisingly :) ) though I do feel that
the simultaneity restrictions could do with a careful re-examination
(particularly with regard to inclusivity vs exclusivity).

Regards,

Andy

On Thu, Dec 13, 2012 at 9:14 PM, Paul Kyzivat <[email-masked]> wrote:

>>> local
> they work on a detailed proposal for how it would work, and then do a full
> assessment of the pros/cons relative to the existing proposal.
>
>         Thanks,
>         Paul
>

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 17 Dec 2012 08:30:55 +0000

Hi,

I am not suggesting that a provider can't provide captures from more than one capture scene entry.

I am suggesting that the provider, instead of indicating which captures it simultaneously can provide, indicate which capture scene entries it can simultaneously provide.

I'll provide more arguments in my reply to Paul's e-mail.

Regards,

Christer

From: [email-masked] [mailto:[email-masked]] On Behalf Of Andy Pepperell
Sent: 14. joulukuuta 2012 19:19
To: Paul Kyzivat
Cc: [email-masked]
Subject: Re: [clue] Can receiver select captures from more than one capture scene entry?

>There have been some suggestions that this could also eliminate the need for simultaneous sets,

While I can see the appeal of this approach, I think we shouldn't discount cases where a consumer might want to configure media captures from more than one capture scene entry within a scene. An MCU, for instance, might want to receive a multi-camera view from a room *and* a single roving camera in order to switch both out to different participants depending on those participants' rendering capabilities. Similarly, a recording device might choose to receive from an MCU both a set of switched loudest speaker captures and a more simple pre-composed view to allow later playback by devices whose capabilities similarly differ.

(also, we planned for capture scene entries to be restricted to a single media type (e.g. audio or video) and so it should perhaps be clarified whether the "shall choose one capture scene entry" should more precisely be "shall choose one capture scene entry of each media type").

>PaulK: IMO the existing (prior?) model was at least clearer, and arguably simpler.

I agree with Paul here! (perhaps unsurprisingly :) ) though I do feel that the simultaneity restrictions could do with a careful re-examination (particularly with regard to inclusivity vs exclusivity).

Regards,

Andy

On Thu, Dec 13, 2012 at 9:14 PM, Paul Kyzivat <[email-masked]<mailto:[email-masked]>> wrote:
The "Capture scene clarifications" topic has been *very* popular, and is getting confusing.

Here I've changed the subject line to focus on one point - a change that was been proposed by Christer. The latest description of it was from Christian Groves:

There have been some suggestions that this could also eliminate the need for simultaneous sets, because the scene entries would *be* the simultaneous sets. I'm not sure if that is an integral part of this proposal or not. (If it is, then it provides part of the justification. Without this part I'm not sure what the benefit is.)

I am inclined to agree that a new definition of advertisement and configuration could be built around such a concept. But to do so I think somebody will need to work out all the details. For instance:
- can the same capture be in more than one capture scene entry?
- how does the configure message specify which entries it wants,
  and which captures from those entries?

IMO the existing (prior?) model was at least clearer, and arguably simpler. The way I have been thinking of it:

The proposed changes would make the configuration message more complex. The receiver would need to enumerate the desired scene entries. And for each selected scene entry, identify those captures that are (or are not) desired.

So, IMO, if some people are interested in this new approach, I suggest they work on a detailed proposal for how it would work, and then do a full assessment of the pros/cons relative to the existing proposal.

        Thanks,
        Paul

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 17 Dec 2012 08:44:44 +0000

Hi,

First, there are a few things that need to be separated:

1) Whether a capture can be part of one or more CSEs. 

2) Advertising/configuring

3) Simultaneous sets

Regarding 1): 

I don't think that question is specific to the issues discussed in this thread, so I'll leave it for now (it needs to be clarified, though).

Regarding 2):

Paul is suggesting: 	we list captures, and for each capture we indicate which CSE(s) the capture is associated with.
I am suggesting: 	we list CSEs, and for each CSE we indicate which captures are associated with the CSE.

Si, it's really about how things are structured :)

If a provider advertises captures, it means that the receiver needs to scan through them, to figure out what CSEs the provider is offering. 

That is not needed if the provider advertises CSEs from the beginning. Of course, the receiver still needs to scan through the advertised CSEs, and see what they contain, but the receiver doesn't need to build the CSE structure.

Regarding 3):

My suggestion is that, instead of indicating which captures can be simultaneously provide, we indicate which CSEs (and, therefore all captures associated with the CSE) can be simultaneously provided.

Something like:

(CSE-X AND CSE-Y) OR CSE-Z.

This means that the provider can provide CSE-X and CSE Y (and all captures associated with CSE-X and CSE-Y) simultaneously, or that the provider can provide CSE-Z (and all captures associated with CSE-Z).

But, again, just because the receiver e.g. chooses to receive CSE-Y, it still doesn't have to receive all captures associated with CSE-Y. It can choose which ones it wants.

Regards,

Christer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Paul Kyzivat
Sent: 13. joulukuuta 2012 23:15
To: [email-masked]
Subject: [clue] Can receiver select captures from more than one capture scene entry?

>> captures. What is locally rendered and how is a local decision."

There have been some suggestions that this could also eliminate the need for simultaneous sets, because the scene entries would *be* the simultaneous sets. I'm not sure if that is an integral part of this proposal or not. (If it is, then it provides part of the justification. 
Without this part I'm not sure what the benefit is.)

The proposed changes would make the configuration message more complex. 
The receiver would need to enumerate the desired scene entries. And for each selected scene entry, identify those captures that are (or are not) desired.

	Thanks,
	Paul

---

From: Simon Pietro Romano <[email-masked]>
Date: Mon, 17 Dec 2012 16:54:54 +0100

Hi,

> This means that the provider can provide CSE-X and CSE Y (and all captures associated with CSE-X and CSE-Y) simultaneously, or that the provider can provide CSE-Z (and all captures associated with…

I like this proposal of expressing constraints as a canonicalized boolean expression.

> But, again, just because the receiver e.g. chooses to receive CSE-Y, it still doesn't have to receive all captures associated with CSE-Y. It can choose which ones it wants.

With respect to this point, we might state that:

1. if a CONFIGURE contains just a reference to a CSE, then all Capture Scenes therein contained are to be received;
2. if a client does not want to receive all of the Capture Scenes contained in a CSE, it has to explicitly indicate the ones it is interested in.

From the data model standpoint, since the same Capture Scene can be contained in multiple CSEs, we might list all Capture Scenes as independent elements (uniquely identifiable through their IDs) and make a reference to them inside each CSE element (through the IDREF approach).

What is your feeling about this?

Simon

> So, IMO, if some people are interested in this new approach, I suggest they work on a detailed proposal for how it would work, and then do a full assessment of the pros/cons relative to the existin…

                     					       _\\|//_
                           				      ( O-O )
   ~~~~~~~~~~~~~~~~~~~~~~o00~~(_)~~00o~~~~~~~~~~~~~~~~~~~~~~~~
                    				Simon Pietro Romano
             				 Universita' di Napoli Federico II
                		     Computer Engineering Department 
	             Phone: +39 081 7683823 -- Fax: +39 081 7683816
                                           e-mail: [email-masked]

		    <<Molti mi dicono che lo scoraggiamento Ë l'alibi degli 
		    idioti. Ci rifletto un istante; e mi scoraggio>>. Magritte.
               			                     oooO
  ~~~~~~~~~~~~~~~~~~~~~~~(   )~~~ Oooo~~~~~~~~~~~~~~~~~~~~~~~~~
					                 \ (            (   )
			                                  \_)          ) /
                                                                       (_/

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 17 Dec 2012 16:17:09 +0000

Hi,

Yes.

> 2. if a client does not want to receive all of the Capture Scenes contained in a CSE, it has to explicitly indicate the ones it is interested in.

Sure. Then, in addition to the CSE reference, it would have to indicate which associated captures it does not want to receive (or, which ones it wants to receive, depending on how we want it).

> What is your feeling about this?

Did you mean to say capture (instead of capture scene), ie that the same capture can be contained in multiple CSEs?

If that's the case, I agree that captures could be listed as independend elements, and then referenced through their IDs.

(If a capture can only be associated with a single CSE, then it can be listed within that CSE).

Regards,

Christer

-----Original Message-----
From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Paul Kyzivat
Sent: 13. joulukuuta 2012 23:15
To: [email-masked]<mailto:[email-masked]>
Subject: [clue] Can receiver select captures from more than one capture scene entry?

Thanks,
Paul

                             _\\|//_
                                  ( O-O )
                                           e-mail: [email-masked]<mailto:[email-masked]>

---

From: Simon Pietro Romano <[email-masked]>
Date: Mon, 17 Dec 2012 17:28:07 +0100

Of course I meant Capture (and not Capture Scene). All these names are driving me nuts!

Simon

                     					       _\\|//_
                           				      ( O-O )

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 17 Dec 2012 12:14:17 -0500

On 12/17/12 10:54 AM, Simon Pietro Romano wrote:
> Hi,
>
>> Regarding 3):
>>
>> My suggestion is that, instead of indicating which captures can be
>> simultaneously provide, we indicate which CSEs (and, therefore all
>> captures associated with the CSE) can be simultaneously provided.
>>
>> Something like:
>>
>> This means that the provider can provide CSE-X and CSE Y (and all
>> captures associated with CSE-X and CSE-Y) simultaneously, or that the
>> provider can provide CSE-Z (and all captures associated with CSE-Z).
>

This is independent of whether this refers to captures or CSEs.

>> But, again, just because the receiver e.g. chooses to receive CSE-Y,
>> it still doesn't have to receive all captures associated with CSE-Y.
>> It can choose which ones it wants.
>
> 1. if a CONFIGURE contains just a reference to a CSE, then all Capture
> Scenes therein contained are to be received;
> 2. if a client does not want to receive all of the Capture Scenes
> contained in a CSE, it has to explicitly indicate the ones it is
> interested in.

I still say that this is simply an optimization on the representation of 
the config message. It is not fundamental.

What is fundamental is a decision that we don't need simultaneous sets 
because CSEs serve the purpose.

Can we please leave the syntax optimizations out of the discussion for 
now? They confuse the discussion of fundamentals.

>  From the data model standpoint, since the same Capture Scene can be
> contained in multiple CSEs, we might list all Capture Scenes as
> independent elements (uniquely identifiable through their IDs) and make
> a reference to them inside each CSE element (through the IDREF approach).

The above makes no sense to me. It would make sense if you replace 
"Capture Scene" with "Capture". Is that what you meant? And yes, I 
generally agree with that.

(I've previously stated that I think captures are scoped to a scene. The 
same capture cannot be in two scenes, because the scene defines the 
coordinate space in which it resides.)

That could be handled by actually nesting the specification of the 
captures within the specification of their Scene. Or it could be handled 
by declaring the captures separately, but including a reference to the 
scene in which they reside. The latter seems a bit cleaner to me.

I have also been thinking about a bigger change: have a single list of 
CSEs for the advertisement, rather than one per scene. (CSE is then a 
misnomer, but for now I'll still use it.) Each entry would contain a 
list of captures that, taken together, is a recommended rendition of the 
advertisement as a whole. This would allow tradeoffs between captures in 
different scenes to offer choices with fewer captures than scenes. There 
is currently no way to do that. I can't think of any downsides to this 
change. What do people think of this?

	Thanks,
	Paul

>> From: [email-masked] <mailto:[email-masked]>
>> [mailto:[email-masked]] On Behalf Of Paul Kyzivat
>> To: [email-masked] <mailto:[email-masked]>
>> Here I've changed the subject line to focus on one point - a change
>> that was been proposed by Christer. The latest description of it was
>> from Christian Groves:
>>
>> There have been some suggestions that this could also eliminate the
>> need for simultaneous sets, because the scene entries would *be* the
>> configuration could be built around such a concept. But to do so I
>> think somebody will need to work out all the details. For instance:
>> The receiver would need to enumerate the desired scene entries. And
>> for each selected scene entry, identify those captures that are (or
>> are not) desired.
>>
> <mailto:[email-masked]>
>

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 17 Dec 2012 18:44:45 +0000

Hi,

Yes.

So, again: My opinion is that we do not need simultaneous capture sets - simulataneous CSE sets are enough.

> generally agree with that.

He meant "yes" :)

> coordinate space in which it resides.)

I agree with that. I think we should keep scenes "independent" from each other, meaning that inside one scene we don't need to refer to stuff in another scenes.

> scene in which they reside. The latter seems a bit cleaner to me.

I agree. The Scene specification should contain the CSEs and captures associated with that Scene:

<scene>
     <CSE list>
            List of CSEs, refering to captures in the capture list
     </CSE list>
     <capture list>
            List of captures
     </capture list>
</scene>

> change. What do people think of this?

I am not sure I understand.

What do you mean by "rather than one per scene"? You can advertise multiple CSEs per scene, can't you?

I am also not sure I understand what you mean by "rendition of the advertisement".

And, I am not sure I understand "tradeoffs between captures in different scenes". I thougth you suggested that captures can only belong to one scene?

Regards,

Christer

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 17 Dec 2012 14:35:44 -0500

> And, I am not sure I understand "tradeoffs between captures in different scenes". I thougth you suggested that captures can only belong to one scene?

OK, obviously I didn't explain myself very well. Let me try again.

First, instead of a list of CSEs per scene, I'm proposing a list of 
(something) per advertisement. Just to have something, for now lets call 
them Advertisement Entries - ADEs. Like a CSE, each ADE is a list of 
captures. In this case they can be any capture in the advertisement, 
regardless of scene.

So overall structure is something like:

<advertisement>
   <scenes>
     <scene id=foo> (descriptive info for the scene) </scene>
     <scene id=bar> (descriptive info for the scene) </scene>
     ...
   </scenes>
   <captures>
     <capture id=f1 scene=foo> ... </capture>
     <capture id=f2 scene=foo> ... </capture>
     <capture id=f3 scene=foo> ... </capture>
     <capture id=b1 scene=bar> ... </capture>
     <capture id=b2 scene=bar> ... </capture>
     <capture id=b3 scene=bar> ... </capture>
     ...
   </captures>
   <ADEs>
     <ADE id=a1>f1,f2,b1,b2</ADE>
     <ADE id=a2>f3,b1,b2</ADE>
     <ADE id=a3>f1,f2,b3</ADE>
     <ADE id=a4>f3,b3</ADE>
     <ADE id=a5>b3</ADE>
   </ADEs>
   </ADEs>
</advertisement>

Here, you can think that f3 is a composite of f1 & f2, and b3 is a 
composite of b1 and b2. Perhaps scene foo is a room, and scene bar 
contains a couple of presentations.

This explicitly allows the advertiser to think about and propose 
tradeoffs between scenes. So a5 makes the decision that if you can only 
receive one capture you are better off getting the presentation than the 
room.

Does it make sense now?

	Thanks,
	Paul

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 17 Dec 2012 20:57:59 +0000

Hi,

> receive one capture you are better off getting the presentation than the

I think it's quite strange if the advertiser starts to advertise "partly scenes", trying to figure out what the receiver may want, and trying to figure out what the capabilities of the receiver are.

My preference is still that we don't need to advertise scene trade-offs. Instead the advertiser provides full scene alternatives (CSEs), and let the receiver choose which scene(s) (and associated CSEs), it wants to receive.

Regards,

Christer

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 17 Dec 2012 17:01:06 -0500

> I think it's quite strange if the advertiser starts to advertise "partly scenes", trying to figure out what the receiver may want, and trying to figure out what the capabilities of the receiver are.

Isn't that what CSEs already do?

Isn't it generally expected that CSEs will differ in the number of 
captures they include, so that they can accommodate consumers that can 
handle only a limited number of captures?

What I have suggested continues that, and extends it to cover multiple 
scenes.

I guess the controversial part in my example is ADE a5, that omits any 
content from scene foo. It has made a value judgement that omitting all 
of foo is still a valid representation of the advertisement. (It might 
be clearer if we had both audio and video, and it retained audio from 
foo but not video.) And it might be that in some cases this wouldn't be 
included in the advertisement because it wouldn't be sufficient.

> My preference is still that we don't need to advertise scene trade-offs. Instead the advertiser provides full scene alternatives (CSEs), and let the receiver choose which scene(s) (and associated C…

I've been talking about this for months. I agree that ultimately it is 
the recipient that makes the final decision. But can be very hard - the 
receiver has limited info to make the decision. This can be partly 
solved by extensions to capture attributes, such as Christian has 
proposed. Including a priority attribute, where priorities have 
significance across scenes, provide another way to accomplish the same 
thing.

Aside from being *different* from what we have assumed for a long time, 
do you see something that makes my proposal worse than having separate 
CSEs for each scene?

(Answering my own question: if there are many alternatives for each 
scene, then my proposal would increase the total number of entries - the 
product of the entries in each. But I doubt this is an issue in practice.)

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 17 Dec 2012 17:13:31 -0500

> I don't think that question is specific to the issues discussed in this thread, so I'll leave it for now (it needs to be clarified, though).

OK.

> I am suggesting: 	we list CSEs, and for each CSE we indicate which captures are associated with the CSE.

No. I was suggesting we list the captures, and for each capture we 
indicate with which *Scene* it is associated. I agree that each CSE 
indicate which captures it contains.

> That is not needed if the provider advertises CSEs from the beginning. Of course, the receiver still needs to scan through the advertised CSEs, and see what they contain, but the receiver doesn't n…

In any case lets not get hung up on syntax. (This is one reason I like 
UML - it divorces the semantics from the syntax.)

The receiver needs to parse the advertisement. It will end up with some 
data structures that may not closely resemble the xml. Regardless of how 
it is encoded, each CSE will reference a bunch of captures that belong 
to the same scene.

This can be represented many different ways.

	Thanks,
	Paul

---

From: Christian Groves <[email-masked]>
Date: Tue, 18 Dec 2012 16:05:04 +1100

Hello Paul,

Please see my comments below.

Regards, Christian

On 14/12/2012 8:14 AM, Paul Kyzivat wrote:
>>> entries
[CNG] I take it that people are OK with the first part of the proposal. 
The only thing in question is "...shall choose one capture scene entry".
> proposal or not. (If it is, then it provides part of the 
> justification. Without this part I'm not sure what the benefit is.)
[CNG] In a later email I proposed a hybrid solution. That we recommend 
that a consumer choose one CSE entry but that it may choose a subset if 
a simultaneous set is included to indicate any dependencies. So instead 
the sentence in question we replace it with

"For a particular media type the consumer should choose one capture 
scene entry. If the advertiser has provided a simultaneous transmission 
set, the consumer may choose individual captures taking in account any 
simultaneity requirements".

> configuration could be built around such a concept. 
[CNG] I don't particularly see this as "new" what I've been trying to do 
is clarify how what is currently in the draft is going to work.

> But to do so I think somebody will need to work out all the details. 
> For instance:
[CNG] The current framework is unclear on this. The examples show that 
captures are only in one capture scene entry at a time. So we need to 
clarify this either way.

>   and which captures from those entries?
[CNG] Again the framework is silent on what happens today. Like the 
scene issue I think people have been making assumptions but these 
haven't been recorded.

> IMO the existing (prior?) model was at least clearer, and arguably 
[CNG] Perhaps but it was not fully documented.

> The way I have been thinking of it:
>
[CNG] I agree. The main thing is that the same logic that was used 
constructing the Advertisement is the same logic that is used "reading" 
the Advertisement.
>   entries. (It might need to identify scenes if the capture ids
[CNG] Generally yes. However there are attributes associated with a 
scene. There may be choices where an attribute is sent with a list where 
a choice could be made. These go beyond just returning an id.
>
>
> The proposed changes would make the configuration message more 
> complex. The receiver would need to enumerate the desired scene 
> entries. And for each selected scene entry, identify those captures 
> that are (or are not) desired.
[CNG] It will really depend on the structure of the message.
> full assessment of the pros/cons relative to the existing proposal.
[CNG] I'd like to nail down how the current framework is meant to work. 
Then we can evaluate any new approach. I think we came up with several 
good clarifications in Marks email on the 02/12.

In addition I think we agreed to add text indicating that no priority 
between elements is intended in the current framework.

Here's some additional items that need clarifying in the current framework:
1. Simultaneous sets: Are these mandatory for the Advertiser to send? If 
not what is the default? What is the action of the consumer?
2. Can a capture occur in multiple capture scene entries?
3. Configure message needs to be specified.

Regards, Christian

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 18 Dec 2012 00:22:25 -0500

Christian,

Fair enough - the "existing" model isn't fully defined either.
So we need to come up with more specifics on it. And then maybe 
comparable specification of candidate alternatives.

If somebody doesn't beat me to it, tomorrow I'll try to post something 
with my understanding of what is intended. (It's too late for me to do 
now and make any sense.)

	Thanks,
	Paul

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 19 Dec 2012 11:43:24 +0000

Hi,

>>>>> I have also been thinking about a bigger change: have a single list 
>>>>> of CSEs for the advertisement, rather than one per scene. (CSE is 
>>>>> then a misnomer, but for now I'll still use it.) Each entry would 
>>>>> contain a list of captures that, taken together, is a recommended 
>>>>> rendition of the advertisement as a whole. This would allow 
>>>>> tradeoffs between captures in different scenes to offer choices 
>>>>> with fewer captures than scenes. There is currently no way to do 
>>>>> that. I can't think of any downsides to this change. What do people think of this?
>>>>
>>> (something) per advertisement. Just to have something, for now lets 
>>> call them Advertisement Entries - ADEs. Like a CSE, each ADE is a 
>>> list of captures. In this case they can be any capture in the 
>>> advertisement, regardless of scene.
>>>
>>> tradeoffs between scenes. So a5 makes the decision that if you can 
>>> only receive one capture you are better off getting the presentation 
>>> than the room.
>>
> Isn't it generally expected that CSEs will differ in the number of captures they include, so that they can accommodate consumers that can handle only a limited number of captures?

Yes, the number of captures will differ. But, each CSE still represents a complete scene.

Or, are you saying that f1 + f2 could be seen as one CSE for foo, and f3 as another CSE for foo (and, the same for b1 + b2 and b3)?

> What I have suggested continues that, and extends it to cover multiple scenes.

My preference would still be to keep scenes separated from each other. The receiver will then, for each scene it wants, choose the scene alternative (CSE) based on its capabilities etc.

> I guess the controversial part in my example is ADE a5, that omits any content from scene foo. It has made a value judgement that omitting 
> all of foo is still a valid representation of the advertisement. (It might be clearer if we had both audio and video, and it retained audio from 
> foo but not video.) And it might be that in some cases this wouldn't be included in the advertisement because it wouldn't be sufficient.
>
> > My preference is still that we don't need to advertise scene trade-offs. Instead the advertiser provides full scene alternatives (CSEs), and 
>> let the receiver choose which scene(s) (and associated CSEs), it wants to receive.
>
> I've been talking about this for months. I agree that ultimately it is the recipient that makes the final decision. But can be very hard - the receiver 
> has limited info to make the decision. This can be partly solved by extensions to capture attributes, such as Christian has proposed. Including a 
> priority attribute, where priorities have significance across scenes, provide another way to accomplish the same thing.

Yes. No matter which way we go, I think the attribute is needed.

> Aside from being *different* from what we have assumed for a long time, do you see something that makes my proposal worse than having separate CSEs for each scene?

One issue is that same as I have had previously: by listing captures, and then having the receiver figure out where they belong (no matter whether we group according to scene, CSE, ADE or whatever). But, again, that is mainly a coding issue.

Another issue is, as indicated above, I see no need for mixing together scenes.

> (Answering my own question: if there are many alternatives for each scene, then my proposal would increase the total number of entries - the product of the entries in each. But I doubt this is an issue in practice.)

I do agree with the fact that the number or alternatives for each scene typically is going to be rather small.

Regards,

Christer

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 19 Dec 2012 11:42:35 -0500

> Or, are you saying that f1 + f2 could be seen as one CSE for foo, and f3 as another CSE for foo (and, the same for b1 + b2 and b3)?

Yes, that is exactly what I meant. If the same case were done with CSEs 
it would be that way.

> My preference would still be to keep scenes separated from each other. The receiver will then, for each scene it wants, choose the scene alternative (CSE) based on its capabilities etc.

I'm trying to discuss the pros/cons on a basis other than mere 
"preference".

> Yes. No matter which way we go, I think the attribute is needed.

Yes, I think so too. But in many cases the attributes will only help a 
human make a manual decision - not sufficient to make an automated 
decision. The CSEs/ADEs lend themselves better to automated selection.
It is particularly hard to make a tradeoff if you can't handle a full 
CSE from each Scene.

> One issue is that same as I have had previously: by listing captures, and then having the receiver figure out where they belong (no matter whether we group according to scene, CSE, ADE or whatever)…

There are multiple potential issues, some more severe than others. I'm 
not sure which you are concerned with. E.g.

- the size of the advertisement

- the complexity of constructing the advertisement

- complexity of *algorithm* for analyzing an advertisement
   and automatically choosing a configuration

- complexity of *algorithm* for analyzing an advertisement
   and constructing a menu of viable alternative configurations
   for an end user to choose among

But IMO what I'm proposing will simplify the configuration algorithms 
for automatic selection of config, may increase the size of the 
advertisement a bit, and is probably neutral for construction of the 
advertisement.

> Another issue is, as indicated above, I see no need for mixing together scenes.

The reason is because the rendering of them will be mixed. The resources 
used to render them can often be used for captures from different 
scenes. Mixing them gives a convenient way for the advertiser to 
recommend potential viable tradeoffs and priorities.

	Thanks,
	Paul

---

From: "Duckworth, Mark" <[email-masked]>
Date: Wed, 19 Dec 2012 15:01:00 -0800

Hi Christian,

I agree with you that a capture scene entry represents  an "entire scene".

I like your proposal for optional simultaneous transmission sets.  I have a suggestion for clarifying 2b and 3b.

2b. if multiple capture scene entries are sent in a scene then the consumer may choose at most one capture scene entry for each media type.

3b. if multiple capture scene entries are sent in a scene then the consumer SHOULD choose one capture scene entry for each media type, but may choose captures based on the simultaneous transmission set.

Mark

> Sent: Wednesday, December 12, 2012 8:53 PM
> To: [email-masked]
> I'm not sure about this. Now we are saying that a capture scene entry does
> not represent an "entire scene" but now represents part of a scene.
> I think logically it works better if an individual capture is part of the scene and
> a capture scene entry depicts an "entire" scene from the viewpoint of the
> provider. If the provider now provides part scenes how does the consumer
> It seems to me that whilst a Consumer may theoretically want to do all sorts
> of things with the captures ultimately its the provider that determines what
> can actually be provided. The consumer may just want to see "the boss" but
> the provider won't know that it has to provide individual captures when it
> does the initial advertisement.
> 
> A provider may be reasonably concise (i.e. I have two capture scene entries
> choose one) or it may be verbose (i.e. I have several capture scene entries, I
> can provide all these different captures, here's the dependency between
> them).
> 
> I've asked previously if simultaneous transmission sets are a mandatory to
> send. I don't think this has been determined.
> 
> 3b. if multiple capture scene entries are sent in a scene then the consumer
> SHOULD close on capture scene entry but may chose captures based on the
> simultaneous transmission set.
> 
> Regards, Christian
> 
> >>>>>>>> entry to be used simultaneously."  So all the media captures from
> a particular capture scene entry must also appear together in a simultaneous
> transmission set.  But that >>>>>simultaneous transmission set could also
> include more media captures, not just media captures that appear in the
> same capture scene entry.
> >>>>>> full scene entry plus one capture from some other scene entry. Or it
> might mean *parts* of two scene entries to assemble something useful that
> always includes the boss.
> >>>>> You can achieve this e.g. using two separate SCENES: one SCENE with
> a capture scene entry that contains the full scene capture, and another
> SCENE with a capture scene entry that contains the boss capture.
> >>>> My assumption was that the boss is not everyone's boss, or known to
> be special in any way to the providing site. Rather he is important only to the
> recipient(s) in one of the other rooms.
> >>>>
> provider's captures that the provider does not realize is of more interest than
> any of its other captures.
> >>> 1) Provide a single SCENE with a capture scene entry that contains the
> "full room", and individual captures of each participant.
> > In my opinion, the receiver would decide two receive both CSEs, but it
> would indicate that from the individual CSE it only wants to "boss" capture.
> >
> > So, the end result is really the same. The only different is that, in my
> opinion, the receiver must always pick a CSE (but it may choose which
> captures it wants).
> >
> > The advantage is that we then only need to advertise simultanous CSE sets
> - not simultanous capture sets - as it is assumed that a provider can always
> provide all captures associated with a CSE.
> >

---

From: Christian Groves <[email-masked]>
Date: Thu, 20 Dec 2012 13:05:41 +1100

Hello Mark,

With regards to 2b is that intent of using "may" that the consumer 
doesn't have to choose any capture?

Regards, Christian

---

From: "Duckworth, Mark" <[email-masked]>
Date: Wed, 19 Dec 2012 20:28:40 -0800

Hello Christian,
Yes, that's right.  Or the consumer may choose a subset of captures from the capture scene entry.
Or the consumer could choose just an audio capture scene entry but no video.
Mark

> Sent: Wednesday, December 19, 2012 9:06 PM
> To: [email-masked]
> With regards to 2b is that intent of using "may" that the consumer doesn't
> have to choose any capture?
> 
> Regards, Christian
> 
> > 2b. if multiple capture scene entries are sent in a scene then the consumer
> may choose at most one capture scene entry for each media type.
> >
> SHOULD choose one capture scene entry for each media type, but may
> choose captures based on the simultaneous transmission set.
> >
> > Mark
> >
> >> the scene and a capture scene entry depicts an "entire" scene from
> >> the viewpoint of the provider. If the provider now provides part
> >> scenes how does the consumer determine what's a full scene?
> >>
> >> It seems to me that whilst a Consumer may theoretically want to do
> >> all sorts of things with the captures ultimately its the provider
> >> that determines what can actually be provided. The consumer may just
> >> want to see "the boss" but the provider won't know that it has to
> >> provide individual captures when it does the initial advertisement.
> >>
> >> I've asked previously if simultaneous transmission sets are a
> >> mandatory to send. I don't think this has been determined.
> >>
> >>>>>>>>>>> what capture scene entries represent. Currently the
> >>>>>>>>>>> framework says that it must be possible to send all captures
> >>>>>>>>>>> in a Capture Scene Entry simultaneously. To me logically
> >>>>>>>>>>> speaking a capture scene entry is effectively a simultaneous
> >>>>>>>>>>> transmission set. To send effectively the same information
> >>>>>>>>>>> twice seems to me
> >> to be abit of a waste.
> >> a particular capture scene entry must also appear together in a
> >> simultaneous transmission set.  But that >>>>>simultaneous
> >> transmission set could also include more media captures, not just
> >> media captures that appear in the same capture scene entry.
> >>>>>>>>> Why can't the "additional captures" be part of a dedicated
> >>>>>>>>> capture
> >> scene entry?
> >>>>>>>> anticipate and advertise every possible combination that a
> >>>>>>>> receiver
> >> might desire.
> >>>>>>>> An example that has been given various times is a recipient
> >>>>>>>> that wants to always receive the capture that includes his
> >>>>>>>> boss, even when the boss isn't speaking.
> >>>>>>>>
> >> a capture scene entry that contains the full scene capture, and
> >> another SCENE with a capture scene entry that contains the boss capture.
> >>>> redundant information, and requires a recipient with no special
> >>>> needs to analyze it and decide it could simply render *either* the "full
> room"
> >>> So, we have a single SCENE, which has two associated CSEs (Capture
> >>> Scene
> >> Entries):
> >>>>> Of the individual participant captures, the receiver will then
> >>>>> only choose the capture associated with the important participant
> >>>>> - ie the receiver is not mandated to receive everything in the
> >>>>> chosen capture scene entry;
> >>>> interests could then choose one of those entries that best meets
> >>>> its needs and capabilities. E.g. if it has enough displays and
> >>>> decoders to receive all the individual captures, then that might be
> >>>> the better choice. If it doesn't have enough resources to do that
> >>>> then it could simply choose the single capture full room entry.
> >>> Agree.
> >>>
> >>>> and also the one capture for the boss from the entry that has all
> >>>> the individual captures.
> >>> Almost agree :)
> >>>
> >> - not simultanous capture sets - as it is assumed that a provider can
> >> always provide all captures associated with a CSE.

---

From: Christian Groves <[email-masked]>
Date: Thu, 20 Dec 2012 16:03:56 +1100

Hello Mark,

OK. Perhaps its worth expanding the text of 2b to explain these 
different possibilities?

Regards, Christian

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 20 Dec 2012 12:30:27 +0000

Hi,

>>>>> only receive one capture you are better off getting the 
>>>>> presentation than the room.
>>>>
> Yes, that is exactly what I meant. If the same case were done with CSEs it would be that way.

In your example, you seem to assume that the receiver wants to receive both scenes (unless it can only receive one). But, maybe the receiver chooses (based on whatever logic) to only receive the "b scene", but still using captures b1 and b2 (and not b3). There is no ADE for that alternative.

The advertiser needs to provide information on what each scene contains, one or more CSEs for each scene, and then let the receiver choose what it wants.

> I'm trying to discuss the pros/cons on a basis other than mere "preference".

I think it's a pro allowing the receiver to choose what it wants to receive, rather than having the provider trying to figure it out. The provider should provide good enough information, allowing the receiver to make the decisions.

>>> I guess the controversial part in my example is ADE a5, that omits 
>>> any content from scene foo. It has made a value judgement that 
>>> omitting all of foo is still a valid representation of the advertisement. (It might be clearer if we 
>>> had both audio and video, and it retained audio from foo but not video.) And it might be that in some cases this wouldn't be included in the advertisement because it wouldn't be sufficient.
>>>
>>>> My preference is still that we don't need to advertise scene 
>>>> trade-offs. Instead the advertiser provides full scene alternatives (CSEs), and let the receiver choose which scene(s) (and associated CSEs), it wants to receive.
>>>
>>> I've been talking about this for months. I agree that ultimately it 
>>> is the recipient that makes the final decision. But can be very hard 
>>> - the receiver has limited info to make the decision. This can be partly solved by extensions to capture attributes, such as 
>>>Christian has proposed. Including a priority attribute, where priorities have significance across scenes, provide another way to accomplish the same thing.
>>
> Yes, I think so too. But in many cases the attributes will only help a human make a manual decision - not sufficient to make an 
>automated decision. The CSEs/ADEs lend themselves better to automated selection. It is particularly hard to make a tradeoff 
>if you can't handle a full CSE from each Scene.

Your "boss" use-case is an example where a manual decision will be needed.

But, in case of automated selections, I think the receiver in most cases will try to get a full scene. The provider will need to provide different alternatives (CSEs) for each scene, and if nothing else works then the receiver probably chooses a CSE containing a single capture for the whole scene (or, it chooses specific captures of a CSE based on e.g. spatial information).

>> One issue is that same as I have had previously: by listing captures, and then having the receiver figure out where they 
>> belong (no matter whether we group according to scene, CSE, ADE or whatever). But, again, that is mainly a coding issue.
>
> There are multiple potential issues, some more severe than others. I'm not sure which you are concerned with. E.g.
>
> But IMO what I'm proposing will simplify the configuration algorithms for automatic selection of config, may increase the size of the advertisement a bit, and is probably neutral for construction of the advertisement.
>
> The reason is because the rendering of them will be mixed. The resources used to render them can often be used for captures from different 
> scenes. Mixing them gives a convenient way for the advertiser to recommend potential viable tradeoffs and priorities.

What simplifies the configuration is providing good information (using scene/capture attributes etc) about what each scene contains, and what individual captures contain, so that the receiver can choose based on its capabilities etc.

Regards,

Christer

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 20 Dec 2012 10:49:23 -0500

> The advertiser needs to provide information on what each scene contains, one or more CSEs for each scene, and then let the receiver choose what it wants.

Either way, IMO the receiver should have the option to request what 
captures it wants. So if it really wants b1 and b2 it is certainly free 
to do so.

What the CSEs (or ADEs in my proposal here) are *recommendations* from 
the provider of choices that will provide satisfactory user experience 
for the resources they consume.

> I think it's a pro allowing the receiver to choose what it wants to receive, rather than having the provider trying to figure it out. The provider should provide good enough information, allowing t…

The two proposals can provide the receiver with an equal amount of 
choice. It is only the hinting/recommendations that are different.

> But, in case of automated selections, I think the receiver in most cases will try to get a full scene. The provider will need to provide different alternatives (CSEs) for each scene, and if nothing…

I agree with everything you say in the sentence above. But that only 
discusses one scene. And in this the provider can help by ensuring that 
there is at least one CSE for the scene that requires only a single display.

But the problem is greater when there is more than one scene. Even if 
each scene has a CSE that requires a single capture, that means the 
minimum number of screens needed to render the full advertisement will 
be equal to the number of screens. (Unless you do local composition or 
switching.) That presents a problem to a receiver that doesn't have that 
many screens.

It is very hard for an algorithm to decide that one of the scenes is 
less important than the other, unless the attributes give the provider's 
assessment of the relative priorities. What my ADE proposal does is give 
the provider a mechanism to recommend tradeoffs between captures in 
different scenes in order to offer combinations that require fewer screens.

> What simplifies the configuration is providing good information (using scene/capture attributes etc) about what each scene contains, and what individual captures contain, so that the receiver can c…

We *could* simply advertise the captures and attributes of the captures, 
without any CSEs. Then let the receiver make its own tradeoffs based on 
the attributes.

I view the CSEs as simply another form of attributes describing the 
captures - a hint of which combinations might provide a satisfactory 
user experience.

So ultimately we are just discussing what representation of provider 
knowledge about the captures will best help the receiver make a good 
choice. And my point is that if the provider has knowledge that can help 
in making tradeoffs between captures in different scenes, then it is 
good to have that.

	Thanks,
	Paul

---

From: "Roni Even" <[email-masked]>
Date: Fri, 21 Dec 2012 00:32:42 +0200

Hi,
We had this discussion from the start. 
My view is that the capture scene entry advertisement provides the
recommended option based on the provider thought about the consumer. Still
the consumer may want to have part of a capture scene entry or any allowed
set based on the simultaneous sets since the consumer may have a different
architecture that is not a good match for any of the proposed capture scene
entries.
As long as we do not have a way to describe the consumer capabilities this
approach will allow him to ask for a better configuration.

Roni

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Paul
Kyzivat
Sent: 13 December, 2012 11:15 PM
To: [email-masked]
Subject: [clue] Can receiver select captures from more than one capture
scene entry?

Here I've changed the subject line to focus on one point - a change that was
been proposed by Christer. The latest description of it was from Christian

>> captures. What is locally rendered and how is a local decision."

There have been some suggestions that this could also eliminate the need for
simultaneous sets, because the scene entries would *be* the simultaneous
sets. I'm not sure if that is an integral part of this proposal or not. (If
it is, then it provides part of the justification. 
Without this part I'm not sure what the benefit is.)

IMO the existing (prior?) model was at least clearer, and arguably simpler.
The way I have been thinking of it:

The proposed changes would make the configuration message more complex. 
The receiver would need to enumerate the desired scene entries. And for each
selected scene entry, identify those captures that are (or are not) desired.

So, IMO, if some people are interested in this new approach, I suggest they
work on a detailed proposal for how it would work, and then do a full
assessment of the pros/cons relative to the existing proposal.

	Thanks,
	Paul

---

From: "Roni Even" <[email-masked]>
Date: Fri, 21 Dec 2012 00:52:43 +0200

Hi,
Here is an example from the framework that can demonstrate the need for
selecting part of capture set entry and captures from multiple capture set
entries

These are the capture scene entries:

1.  (VC0, VC1, VC2) - left, center and right camera video captures

   2.  (VC3) - video capture associated with loudest room segment

   3.  (VC4) - video capture zoomed out view of all people in the room

The simultaneous sets is

(VC0,VC1,VC2,VC3) - since the loudest speaker is one of VC0, VC1 or VC2
(VC0, VC3, VC2) - the middle camera is used for zoom out view.

Now an MCU may want to get (VC0, VC1, VC2 , VC3) and switch to one monitor
systems VC3 and to the 3 monitor systems (VC0,VC1,VC2)

A two monitor system may want to receive (VC4, VC0) since the person using
the system would like to have also a view of the left side of the room

Again, My claim is that the configuration may be selected by the person/s
using the system and the consumer endpoint can provide him with the allowed
multiple choices allowing him to build the view he wants.

Roni

-----Original Message-----

	Thanks,
	Paul

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 21 Dec 2012 12:36:02 +0000

Hi,

>>>>>>> Here, you can think that f3 is a composite of f1 & f2, and b3 is 
>>>>>>> a composite of b1 and b2. Perhaps scene foo is a room, and scene 
>>>>>>> bar contains a couple of presentations.
>>>>>>>
>>>>>>> tradeoffs between scenes. So a5 makes the decision that if you 
>>>>>>> can only receive one capture you are better off getting the 
>> In your example, you seem to assume that the receiver wants to receive both scenes (unless it can only receive one). But, maybe 
>> the receiver chooses (based on whatever logic) to only receive the "b scene", but still using captures b1 and b2 (and not b3). There 
>> is no ADE for that alternative.
>>
> Either way, IMO the receiver should have the option to request what captures it wants. So if it really wants b1 and b2 it is certainly free to do so.
>
> What the CSEs (or ADEs in my proposal here) are *recommendations* from the provider of choices that will provide satisfactory user experience for the resources they consume.

I don't agree that a CSE is a recommendation. A CSE is a technical description for providing a full scene.

>>> The two proposals can provide the receiver with an equal amount of choice. It is only the hinting/recommendations that are different.
>>
>>>>>> omitting all of foo is still a valid representation of the 
>>>>> advertisement. (It might be clearer if we had both audio and video, and it retained audio from foo but not video.) And it might be that in some cases this wouldn't be included in the advertisement because it wouldn't be sufficient.
>>>>>
>>>>> - the receiver has limited info to make the decision. This can be 
>>>>> partly solved by extensions to capture attributes, such as Christian has proposed. Including a priority attribute, where priorities have significance across scenes, provide another way to accomplish the same thing.
>>>>
>>> Yes, I think so too. But in many cases the attributes will only help 
>>> a human make a manual decision - not sufficient to make an automated 
>>> decision. The CSEs/ADEs lend themselves better to automated selection. It is particularly hard to make a tradeoff if you can't handle a full CSE from each Scene.
>>
>> But, in case of automated selections, I think the receiver in most cases will try to get a full scene. The provider will need to provide different 
>> alternatives (CSEs) for each scene, and if nothing else works then the receiver probably chooses a CSE containing a single capture for the whole 
>> scene (or, it chooses specific captures of a CSE based on e.g. spatial information).
>
> I agree with everything you say in the sentence above. But that only discusses one scene. And in this the provider can help by ensuring that there is at least one CSE for the scene that requires only a single display.

The provider can do that by providing a CSE with a single capture.

> But the problem is greater when there is more than one scene. Even if each scene has a CSE that requires a single capture, that means the 
> minimum number of screens needed to render the full advertisement will be equal to the number of screens. (Unless you do local composition or
> switching.) That presents a problem to a receiver that doesn't have that many screens.
>
> It is very hard for an algorithm to decide that one of the scenes is less important than the other, unless the attributes give the provider's assessment 
> of the relative priorities. What my ADE proposal does is give the provider a mechanism to recommend tradeoffs between captures in different scenes 
> in order to offer combinations that require fewer screens.

CSEs give you the same options, because they can provide the scene with different number of captures etc. And, if the receiver can only choose one scene, it chooses the one it thinks is most important.

There needs to be some information associated with each scene, what it represents.

In typical scenarios there will be two scenes, a "main" scene, and a "presentation" scene, so there is no need for a complex algorithm. 

>>>> One issue is that same as I have had previously: by listing 
>>>> captures, and then having the receiver figure out where they belong (no matter whether we group according to scene, CSE, ADE or whatever). But, again, that is mainly a coding issue.
>>>
>>> The reason is because the rendering of them will be mixed. The 
>>> resources used to render them can often be used for captures from different scenes. Mixing them gives a convenient way for the advertiser to recommend potential viable tradeoffs and priorities.
>>
> We *could* simply advertise the captures and attributes of the captures, without any CSEs. Then let the receiver make its own tradeoffs based on the attributes.

The advantage of CSEs is that we don't need simultaneous capture sets.

...and, the receiver does not need to parse and generate the CSEs itself. It may not be a problem when there is only one alternative for providing a scene, but if there are many alternatives the receiver needs to figure out both which scene each capture belongs to, but also which captures are needed in order to receive a full scene.

> I view the CSEs as simply another form of attributes describing the captures - a hint of which combinations might provide a satisfactory user experience.
>
> So ultimately we are just discussing what representation of provider knowledge about the captures will best help the receiver make a good 
> choice. And my point is that if the provider has knowledge that can help in making tradeoffs between captures in different scenes, then it is good to have that.

Again, I don't view them as a hint or recommendation. They describe how a full scene can be provided, using a particular number of captures etc.

So, we do need some kind of "scene attribute". And, in some cases we also need a "capture attribute", e.g. if a capture is associated with a given individual.

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 21 Dec 2012 12:40:34 +0000

Hi,

I thought each capture scene entry (CSE) represented the same scene. I guess 1) and 3) represents the same scene, but not 2) is only part of the same scene.

Regards,

Christer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Roni Even
Sent: 21. joulukuuta 2012 0:53
To: 'Paul Kyzivat'; [email-masked]
Subject: Re: [clue] Can receiver select captures from more than one capture scene entry?

Hi,
Here is an example from the framework that can demonstrate the need for selecting part of capture set entry and captures from multiple capture set entries

The simultaneous sets is

(VC0,VC1,VC2,VC3) - since the loudest speaker is one of VC0, VC1 or VC2 (VC0, VC3, VC2) - the middle camera is used for zoom out view.

Now an MCU may want to get (VC0, VC1, VC2 , VC3) and switch to one monitor systems VC3 and to the 3 monitor systems (VC0,VC1,VC2)

A two monitor system may want to receive (VC4, VC0) since the person using the system would like to have also a view of the left side of the room

Again, My claim is that the configuration may be selected by the person/s using the system and the consumer endpoint can provide him with the allowed multiple choices allowing him to build the view he wants.

Roni

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Paul Kyzivat
Sent: 13 December, 2012 11:15 PM
To: [email-masked]
Subject: [clue] Can receiver select captures from more than one capture scene entry?

Here I've changed the subject line to focus on one point - a change that was been proposed by Christer. The latest description of it was from Christian

	Thanks,
	Paul

---

From: "Roni Even" <[email-masked]>
Date: Fri, 21 Dec 2012 15:48:04 +0200

Hi Christer,
2 represent a switch view of the scene, this is what will be offered from a
three camera system to a one decoder/monitor system.
This is from the framework
Roni

-----Original Message-----
From: Christer Holmberg [mailto:[email-masked]] 
Sent: 21 December, 2012 2:41 PM
To: Roni Even; 'Paul Kyzivat'; [email-masked]
Subject: RE: [clue] Can receiver select captures from more than one capture
scene entry?

Hi,

I thought each capture scene entry (CSE) represented the same scene. I guess
1) and 3) represents the same scene, but not 2) is only part of the same
scene.

Regards,

Christer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Roni
Even
Sent: 21. joulukuuta 2012 0:53
To: 'Paul Kyzivat'; [email-masked]
Subject: Re: [clue] Can receiver select captures from more than one capture
scene entry?

Hi,

The simultaneous sets is

Roni

-----Original Message-----

	Thanks,
	Paul

---

From: "Duckworth, Mark" <[email-masked]>
Date: Fri, 21 Dec 2012 15:54:28 -0800

I agree we should keep the ability for a consumer to select captures from multiple capture scene entries.  But I also like Christian's hybrid proposal, because it enables both providers and consumers to be simple if they want to be.

With Christian's proposal, a simple provider can omit the simultaneous transmission sets, and then the capture scene entries become the sole factor that defines what the provider can send simultaneously.  In this case, a consumer cannot select captures from more than one capture scene entry (of same media type).  A provider that wants to offer more choice to the consumer can use simultaneous transmission sets to give the consumer more information, and this enables the consumer to choose captures from more than one capture scene entry.

A simple consumer can always ignore simultaneous transmission sets, and simply choose captures from only one capture scene entry, typically choosing all the captures in a capture scene entry.  A more complicated consumer could take advantage of the additional choices available when simultaneous transmission sets are used by the provider.

Mark

> Roni Even
> Sent: Thursday, December 20, 2012 5:53 PM
> Now an MCU may want to get (VC0, VC1, VC2 , VC3) and switch to one
> monitor systems VC3 and to the 3 monitor systems (VC0,VC1,VC2)
> 
> using the system and the consumer endpoint can provide him with the
> allowed multiple choices allowing him to build the view he wants.
> 
> Roni
> 
> Paul Kyzivat
> simultaneous sets. I'm not sure if that is an integral part of this proposal or
> not. (If it is, then it provides part of the justification.
> complex.

---

From: "Duckworth, Mark" <[email-masked]>
Date: Fri, 21 Dec 2012 16:07:16 -0800

Hi Christian,

I'll take a few of the issues from below and give my input.

> > - can the same capture be in more than one capture scene entry?
Yes.  We should clarify this in the framework.

>   and which captures from those entries?
Section 9 of the framework says "the consumer must choose which media captures it wishes to receive, and which individual encodings from the provider it wants to use to encode the captures."  Then there is a bit more detail.  This just means the consumer specifies which media captures it wants to receive, with information about how it is encoded.  The consumer does not need to mention capture scene entries in the configure message, it just lists the media captures it wants.  This is simple, and it can be the same whether or not simultaneous transmission sets are involved.  I think any further detail about how all this is represented belongs in a protocol document, not the framework.

> not what is the default? What is the action of the consumer?
I think you hybrid proposal is good, as I explained in separate emails.

Regards,
Mark

> Sent: Tuesday, December 18, 2012 12:05 AM
> To: Paul Kyzivat
> Cc: [email-masked]
> >>> media captures are spatially related. For a particular media type
> >>> the consumer shall choose one capture scene entry rather than
> >>> choosing individual captures from multiple capture scene entries.
> >>> However this does not mean that a consuming endpoint must render all
> >>> the media captures. What is locally rendered and how is a local
> >>> decision."
> [CNG] In a later email I proposed a hybrid solution. That we recommend that
> a consumer choose one CSE entry but that it may choose a subset if a
> simultaneous set is included to indicate any dependencies. So instead the
> sentence in question we replace it with
> 
> "For a particular media type the consumer should choose one capture scene
> entry. If the advertiser has provided a simultaneous transmission set, the
> consumer may choose individual captures taking in account any simultaneity
> requirements".
> 
> 
> >
> [CNG] I don't particularly see this as "new" what I've been trying to do is
> clarify how what is currently in the draft is going to work.
> 
> captures are only in one capture scene entry at a time. So we need to clarify
> this either way.
> 
> [CNG] Again the framework is silent on what happens today. Like the scene
> issue I think people have been making assumptions but these haven't been
> recorded.
> 
> >