Subject: Re: [clue] Use Case and Framework question

From: Christian Groves <[email-masked]>
Date: Thu, 01 Sep 2011 17:03:00 +1000

Hello Marshall,

I think your suggested method is worth exploring. Some comments, 
questions and thinking out loud below marked CNG.

Regards, Christian

On 31/08/2011 10:07 PM, Marshall Eubanks wrote:
>
>
> On Wed, Aug 31, 2011 at 6:25 AM, Christian Groves 
> <[email-masked] <mailto:[email-masked]>> 
> wrote:
>
>     Hello,
>
>     Whilst reviewing the use case document use case 4.1 point to point
>     meeting talks about the possibility for separate monophonic audio
>     streams. Now this presumably this allows for a microphone to be
>     placed in front of each participant.
>
>     Now consider a three position (left to right) two row telepresence
>     system. Each of these has a microphone i.e. 6 audio captures,
>     Front row AC0, AC1, AC2, Back row AC3, AC4, AC5.
>
>     The current framework document considers video and audio captures
>     to be left to right.
>     So its easy to describe the first row AC0, AC1, AC2. How would I
>     describe using the current framework the audio captures for the
>     second row microphones?
>
>     Even if we disregard the row (depth) element, how would I then say
>     AC0 & AC3, AC1 & AC4 and AC3 & AC5 relate to the same left /
>     centre / right position?
>
>
> My own personal opinion is that
>
> - it is dangerous and limiting to attempt to capture equipment spatial 
> placement in a simple ordered list, especially when that equipment (or 
> its capture zone) can move and change with time and that
>
> - we are likely to be sending kilobytes of information about codec 
> formats and the like , so I don't see why we can't afford a few bytes to
> describe a mapping between equipment ids (numbering) and what and 
> where they are capturing.
[CNG] I would agree it seems bytes are cheap these days...
>
> If you want a specific proposal to deal with this, here is one.
>
> There are three possible rankings of spatial equipment capture, 
> transverse, depth and vertical. Call them T, D and V.
>
> Each can be declared to be relevant, with the assumption that at least 
> one will be. Ones not declared relevant are ignored.
>
> Each can be declared to be fixed or variable width. In the case of 
> fixed width, the width is also declared.
>
> Units are degrees (T AND V) and meters (D).
>
> Everything is referenced to the axis of symmetry of the unit, oriented 
> from the point of view of an observer standing on the axis at the 
> capture equipment facing the participants (i.e., the point of view of 
> an outside observer). Order is right to left (T), front to back (D) 
> and bottom to top (V) . A "starting" value is given (so that we don't 
> have to deal with negative numbers in the list) and a "unit" value 
> (the smallest addressable chunk of degrees or meters).
[CNG] Just to be 100% sure when you say axis of symmetry of "the unit" 
do you mean each media capture device or the telepresence system (TPS)? 
I guess when we start to take into geometries into consideration the 
description (and implementation) does become more complex. It might get 
interesting to consider a presentation related capture.

Did the framework draft authors consider such an approach when drafting 
the draft text?

>
> All spatially equipment is given an ID (could just be numbered, it 
> doesn't really matter).
>
> So, assuming the above AC0-AC5, this situation could be dealt with as 
> something like
>
> Declare T UNIT 30 STARTING -60
> Declare D UNIT 2 STARTING 2
[CNG] Are these fixed as described above? If so is the width declared?

>
> Order (ACO, 1,2), (AC1, 2, 2), (AC2, 3,2), (AC3, 1,3), (AC4, 
> 2,3), (AC5, 3,3),
>
> where (ACO, 1,1) means unit ACO covers T zone 1 (-60 to -30 degrees) 
> and D zone 2 (2 to 4 meters into the depth of field), etc.
[CNG] I'm a bit confused with this example given the declares. Do you 
mean (ACO, 1,2)? or should AC0 be changed in the ordered list?
       Given a triplet (AC0,t,d)  t=1, d=1
       AC0 is the identity
       t is a multiplier of the unit T in this case 30deg added to the 
start -60,
            i.e. (t * 30) + (-60) = -30
       d is a multiplier of the unit D in this case 2m added to the start 2,
            i.e. (d * 2) + (2) = 4

In words how would you describe? (AC4,2,3)  e.g. t=2, d=3

>
> If there was a camera that covered all 120 degrees, and 3 others that 
> covered 30 degrees each, that could be
>
> Declare T VARIABLE UNIT 30 STARTING -60
>
> Order (VC0, 1, 4), (VC1, 1, 1), (VC1, 1, 1), (VC2, 2, 1), (VC3, 3, 1),
>
> where (VC0, 1, 4) means unit VC0 covers from -60 to +60 degrees (4 30 
> degree slices), unit VC1 covers -60 to -30 degrees (1 slice), etc.
[CNG] How would you handle overlapping regions?

[CNG] Would you see it being problematic if rather than specifying this 
in terms of units and multipliers that the start and end for each of the 
variables be explicitly used?
e.g. Based on T,D,V
For video
      (VC0,[-60:60]),(VC1,[-60:-30])?
or in the audio capture case?
      (AC0,[-60:-30],[2:4]),(AC4,[-30:0],[4:6])

>
> I don't see this as really much more complicated or byte consuming and 
> it really would avoid a bunch of problems I see as coming from 
> overlaying a potentially variable 3-D spatial order onto a simple list.
[CNG] Perhaps another approach is? to treat each capture set as having 
"a spatial relationship" between media captures however there is no 
significance of the position of the media capture in the capture set 
definition itself (i.e. remove all spatial order from the list). The 
spatial details related to the capture are given as attributes of each 
media capture, e.g. T, D, V or other as appropriate.

>
> Regards
> Marshall
>
>     Regards, Christian
>
>     _______________________________________________
>     clue mailing list
>     [email-masked] <mailto:[email-masked]>
>     https://www.ietf.org/mailman/listinfo/clue
>
>

---

From: Stephen Botzko <[email-masked]>
Date: Thu, 1 Sep 2011 08:09:57 -0400

I guess in the limit the safe path is to include a dimensioned engineering
drawing of the room in the signaling, showing placement of every sensor,
rendering device, and the furniture, and providing real-time coordinate
updates for each movable device.

Part of the challenge here is to identify the core information that is
actually useful to receivers and senders in establishing a good experience
between unlike systems, and also to consider what information is actually
known by real systems (which often are installed by third party integrators,
or are modified after installation, have no means to precisely determine
where a handheld camera is aimed, etc). In my view there is a considered
trade-off that will have to be made between completeness, complexity, and
deployment constraints.

This is not to say that we shouldn't review alternatives, or that the
submitted framework draft has everything needed.  I am just saying that when
I work through the use cases with different room designs, I am finding that
a lot of the detailed information we could potentially provide is not very
useful in creating an interoperable experience.  I am also finding that it
is very important to structure the information so that it is easy to extract
the core parameters you actually need.

So I would encourage folks who want to improve the framework (or develop
their own alternatives) to attempt to apply their proposals to the use cases
- a lot of things that initially seemed important to me didn't turn out to
be helpful when I did that.

Stephen Botzko

On Wed, Aug 31, 2011 at 8:07 AM, Marshall Eubanks <
[email-masked]> wrote:

>
>
> On Wed, Aug 31, 2011 at 6:25 AM, Christian Groves <
> [email-masked]> wrote:
>
>> Hello,
>>
>> Whilst reviewing the use case document use case 4.1 point to point meeting
>> talks about the possibility for separate monophonic audio streams. Now this
>> presumably this allows for a microphone to be placed in front of each
>> participant.
>>
>> Now consider a three position (left to right) two row telepresence system.
>> Each of these has a microphone i.e. 6 audio captures, Front row AC0, AC1,
>> AC2, Back row AC3, AC4, AC5.
>>
>> The current framework document considers video and audio captures to be
>> left to right.
>> So its easy to describe the first row AC0, AC1, AC2. How would I describe
>> using the current framework the audio captures for the second row
>> microphones?
>>
>> Even if we disregard the row (depth) element, how would I then say AC0 &
>> AC3, AC1 & AC4 and AC3 & AC5 relate to the same left / centre / right
>> position?
>>
>>
> placement in a simple ordered list, especially when that equipment (or its
> capture zone) can move and change with time and that
>
> - we are likely to be sending kilobytes of information about codec formats
> and the like , so I don't see why we can't afford a few bytes to
> describe a mapping between equipment ids (numbering) and what and where
> they are capturing.
>
> There are three possible rankings of spatial equipment capture, transverse,
> depth and vertical. Call them T, D and V.
>
> Each can be declared to be relevant, with the assumption that at least one
> will be. Ones not declared relevant are ignored.
>
> Each can be declared to be fixed or variable width. In the case of fixed
> width, the width is also declared.
>
> Everything is referenced to the axis of symmetry of the unit, oriented from
> the point of view of an observer standing on the axis at the capture
> equipment facing the participants (i.e., the point of view of an outside
> observer). Order is right to left (T), front to back (D) and bottom to top
> (V) . A "starting" value is given (so that we don't have to deal with
> negative numbers in the list) and a "unit" value (the smallest addressable
> chunk of degrees or meters).
>
> All spatially equipment is given an ID (could just be numbered, it doesn't
> really matter).
>
> Order (ACO, 1,2), (AC1, 2, 2), (AC2, 3,2), (AC3, 1,3), (AC4, 2,3), (AC5,
> 3,3),
>
> where (ACO, 1,1) means unit ACO covers T zone 1 (-60 to -30 degrees) and D
> zone 2 (2 to 4 meters into the depth of field), etc.
>
> I don't see this as really much more complicated or byte consuming and it
> really would avoid a bunch of problems I see as coming from overlaying a
> potentially variable 3-D spatial order onto a simple list.
>
> Regards
> Marshall
>
>
>
>> Regards, Christian
>>
>> ______________________________**_________________
>> clue mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/**listinfo/clue<https://www.ietf.org/mailman/listinfo/clue>
>>
>
>
> [email-masked]

---

From: Stephen Botzko <[email-masked]>
Date: Thu, 1 Sep 2011 10:53:03 -0400

in-line

On Thu, Sep 1, 2011 at 12:19 AM, Christian Groves <
[email-masked]> wrote:

> Hello Stephen,
>
> Please see my responses below.
>
> Regards,
> Christian
>
>
>
> On 31/08/2011 9:47 PM, Stephen Botzko wrote:
>
>> The current framework draft allows multiple captures to signal the same
>> linear index.  So it is fine for AC0,AC3 and the the other pairs to all
>> signal the same index.  This covers the case where the two rows are captured
>> in the same video captures (that is, the left camera VC0 captures the left
>> side of both rows). If the video capture set is VC0, VC1, VC2,  it is
>> possible to associate both AC3 and AC0 with VC0.
>>
> [CNG] OK, linear index is used to indicate the position of an audio capture
> rather than the left/right behaviour?

[SCB]Yes.  That makes it easy to match the sound stage to the displays, and
also deals nicely with the situation where the captures are not evenly
placed in the room.

>
>
>
>> There is a related case, which is when there are also independent cameras
>> (AC3 has a corresponding VC3).  The framework also allows multiple capture
>> sets, so you can signal VC3, VC4, VC5, AC3, AC4, AC5 as its own independent
>> capture set.
>>
> [CNG] However in this case you would need some "depth" parameter/s?
> [SCB]  That certainly is one option.  For many meeting situations receivers
> don't really need to know that VC0 is row 1, and VC3 is row 2; but I suspect
> there are cases (perhaps educational) where you might want to know this.
>
>
>> There's been quite a bit of discussion on how two "separate monophonic
>> streams" relate to a "stereo audio stream" amongst the authors of the
>> framework.  At this point I believe the conclusion is that the key
>> distinction between them is RTP transport.  A second possible distinction is
>> that some audio codecs allow a stereo audio stream to be jointly encoded,
>> providing somewhat better compression.
>>
> [CNG] I had assumed that the monophonic audio streams were separate RTP
> streams. However my question was more looking at the limitation of the
> framework model rather than opening how audio is encoded and carried.

[SCB] Understood.

>
>
>
>> BTW, though it is convenient to think about audio captures as independent
>> microphones, it is important to keep in mind that there are other ways to
>> generate them.  For instance, a microphone array can use beam-forming
>> techniques to construct multiple captures from the microphone array.  This
>> can in principle be done with video, particularly if you also have depth
>> information. The framework does not describe how many sensors are used to
>> create a capture.
>>
> [CNG] I agree. My questions came as a result of the use cases which talk
> about monophonic stream and microphones. In the beam-forming would the
> aspect of "depth" in addition to "linear index" also be useful?

> [SCB] Though I don't think you are suggesting this, I'd like to say that I
> don't think we need to send all the information needed for receivers to form
> the beam, it is much more practical to have senders form it.

> I think sorting out how precisely/completely we need to describe the useful
> pickup pattern of an audio or video camera needs more discussion.  For audio
> particularly, sound pickup depends on the sensor array, the room design, and
> who/what is in the room at the moment.  I don't think its helpful [and often
> not even possible] to describe it too exactly, but it is true that we might
> need more than a single linear position. One helpful way to sort out what is
> needed is to think about the rendering constraints.
>

> For instance, in almost all non-mobile cases the audio is being rendering
> through a linear array of front speakers.  If rear speakers are used, they
> are generally for sound reinforcement - meaning that they are adjusted so
> that the Haas effect ensures that the sound is always perceived as coming
> from the front of the room.   In that common situation, the one dimensional
> linear index (a panning position) is all you really need.  Adding more
> parameters will not improve the spatial rendering much.  Even if you are not
> rendering with a simple linear array, creating spatial audio with depth in
> telepresence rooms means placing the far-end sound *behind* the monitors -
> and adding rear speakers to the room doesn't help you do that. I don't know
> any good way to render room audio so it sounds like it is coming from an
> arbitrary depth behind a wall-mounted monitor or projector screen.
>

> However, if a middle box is selecting a subset of the audio captures in a
> room, then it conceivably might be necessary to provide more information
> about the pickup region.  So far I haven't come up with good use case that
> shows that though.
>

>> [email-masked] <mailto:Christian.Groves@**nteczone.com<[email-masked]>>>
>> wrote:
>>
>>    Hello,
>>

---

From: Marshall Eubanks <[email-masked]>
Date: Thu, 1 Sep 2011 12:29:22 -0400

Dear Christian;

On Thu, Sep 1, 2011 at 3:03 AM, Christian Groves <
[email-masked]> wrote:

> Hello Marshall,
>
> I think your suggested method is worth exploring. Some comments, questions
> and thinking out loud below marked CNG.
>
> Regards, Christian
>
>> from the point of view of an observer standing on the axis at the capture
> [CNG] Just to be 100% sure when you say axis of symmetry of "the unit" do
> you mean each media capture device or the telepresence system (TPS)? I guess
> when we start to take into geometries into consideration the description
> (and implementation) does become more complex. It might get interesting to
> consider a presentation related capture.
>

In many physics problems, determining the appropriate frame of reference is
half of the battle, and I suspect that this will be the case here as well.

I find it useful to use the imagery and nomenclature of stagecraft in
videoconferencing and telepresence. There is, of course, a rather longer
history of running code in the theater and related arts. In the theater, as
in videoconferencing and telepresence, there is a local audience. There are
also performers (in theater) or remote participants (in telepresence). Think
about the most basic things in that situation

- There is a spatial division between the audience and the performers. The
audience is here, the performers are over there.

- The audience sits and looks in some direction. They may look left or
right, up or down, but they almost never have to turn around and look behind
them. The performers occupy in the forward directions they can look at
comfortably.

- There is almost  always a default direction (what the audience looks at if
they look straight ahead) which is an axis of symmetry for the entire room.
(That is true even for theater in the round.)

This basic setup is seen across cultures, in theater, music, teachers in
schools, religious ceremonies, judges in courtrooms, etc. It was familiar to
the ancient Greeks, not to mention the Chinese, Japanese, etc. It is also
the basic setup for all of the telepresence solutions I know of. There is so
much running code here that if we can accommodate it, we are probably
"future-proof" for some time to come. That is what I think we should aim
for, and that is what I was trying to do.

So, I use the axis of symmetry as the basic reference point. Think of it as,
where the audience is looking. If someone can come up with a better
reference point, I am not wed to it, but I would want to accommodate all of
the above cases.

>
> Did the framework draft authors consider such an approach when drafting the
> draft text?
>
>
I have urged this verbally but hadn't previously put in writing.

> [CNG] Are these fixed as described above? If so is the width declared?
The width is UNIT. I like WIDTH better.

so

1,$s/UNIT/UNIT_WIDTH/

> [CNG] I'm a bit confused with this example given the declares. Do you mean
> (ACO, 1,2)? or should AC0 be changed in the ordered list?
>      t is a multiplier of the unit T in this case 30deg added to the start
> -60,
Note that there is a typo here - VC

> [CNG] Would you see it being problematic if rather than specifying this in
> terms of units and multipliers that the start and end for each of the
>     (VC0,[-60:60]),(VC1,[-60:-30])**?
>     (AC0,[-60:-30],[2:4]),(AC4,[-**30:0],[4:6])
>
>
>
> [CNG] Perhaps another approach is? to treat each capture set as having "a
> spatial relationship" between media captures however there is no
> definition itself (i.e. remove all spatial order from the list). The spatial
> details related to the capture are given as attributes of each media
> capture, e.g. T, D, V or other as appropriate.
>
>
>
>
>
>> Regards
>> Marshall
>>
>>    Regards, Christian
>>

---

From: Marshall Eubanks <[email-masked]>
Date: Thu, 1 Sep 2011 12:57:44 -0400

Dear Christian;

so

1,$s/UNIT/UNIT_WIDTH/

>> Order (VC0, 1, 4), (VC1, 1, 1), (VC1, 1, 1), (VC2, 2, 1), (VC3, 3, 1),
Note that there is a typo here - VC1 is repeated.

> [CNG] How would you handle overlapping regions?
Just do it. The above is overlapping - VC0 overlaps VC1, 2 and 3.

Or, if you mean, how should the consumer handle overlapping regions - I
don't see that as our job. We set up the infrastructure to describe what's
going on. Maybe the consumer only has one screen for this remote site, so
they use VC0. Maybe they have lots of bandwidth, but only one screen, so
instead they monitor VC0 and use it to pick between VC1, 2 or 3. Maybe they
crop overlaps. Whatever. IMO we shouldn't be specifying that.

>     (AC0,[-60:-30],[2:4]),(AC4,[-**30:0],[4:6])
Sure. I do not have a good understanding as to what the vendors are
comfortable with here - whether they want very simple commands or the full
granularity. They should drive that aspect here IMO.

I said before that some relation (T or V or D) should be assumed to be
present, but it came to me that a one camera mono sound unit doesn't need
this, so that could be assumed if the ordering was absent.

> capture, e.g. T, D, V or other as appropriate.
I am not sure I am parsing that correctly. Could you provide an example ?

Regards
Marshall

---

From: Christian Groves <[email-masked]>
Date: Fri, 02 Sep 2011 10:42:49 +1000

Hello Stephen,

In-line below

Regards, Christian

On 2/09/2011 12:53 AM, Stephen Botzko wrote:
> in-line
>
> On Thu, Sep 1, 2011 at 12:19 AM, Christian Groves 
>         The current framework draft allows multiple captures to signal
>         the same linear index.  So it is fine for AC0,AC3 and the the
>         other pairs to all signal the same index.  This covers the
>         case where the two rows are captured in the same video
>         captures (that is, the left camera VC0 captures the left side
>         of both rows). If the video capture set is VC0, VC1, VC2,  it
>         is possible to associate both AC3 and AC0 with VC0.
>
>     [CNG] OK, linear index is used to indicate the position of an
>     audio capture rather than the left/right behaviour?
>
> [SCB]Yes.  That makes it easy to match the sound stage to the 
> displays, and also deals nicely with the situation where the captures 
> are not evenly placed in the room.
[CNG] Rather than having to capture set left->right convention and the 
linear index could we just have the linear index? i.e. the convention is 
removed. If we have both then don't we run into potential inconsistency 
problems. e.g. (AC0[lin_ind=30],AC1[lin_ind=90],AC2[lin_ind=60])?

..snip..
>
>
>
>
>
>         BTW, though it is convenient to think about audio captures as
>         independent microphones, it is important to keep in mind that
>         there are other ways to generate them.  For instance, a
>         microphone array can use beam-forming techniques to construct
>         multiple captures from the microphone array.  This can in
>         principle be done with video, particularly if you also have
>         depth information. The framework does not describe how many
>         sensors are used to create a capture.
>
>     [CNG] I agree. My questions came as a result of the use cases
>     which talk about monophonic stream and microphones. In the
>     beam-forming would the aspect of "depth" in addition to "linear
>     index" also be useful?
>
>     [SCB] Though I don't think you are suggesting this, I'd like to
>     say that I don't think we need to send all the information needed
>     for receivers to form the beam, it is much more practical to have
>     senders form it. 
>
>     I think sorting out how precisely/completely we need to describe
>     the useful pickup pattern of an audio or video camera needs more
>     discussion.  For audio particularly, sound pickup depends on the
>     sensor array, the room design, and who/what is in the room at the
>     moment.  I don't think its helpful [and often not even possible]
>     to describe it too exactly, but it is true that we might need more
>     than a single linear position. 
>
[CNG] I agree. I think the challenge is to keep the common cases simple 
whilst allowing the more complex cases. It my mean we need different but 
complementary ways of describing this.

>     One helpful way to sort out what is needed is to think about the
>     rendering constraints.
>
>     For instance, in almost all non-mobile cases the audio is being
>     rendering through a linear array of front speakers.  If rear
>     speakers are used, they are generally for sound reinforcement -
>     meaning that they are adjusted so that the Haas effect ensures
>     that the sound is always perceived as coming from the front of the
>     room.   In that common situation, the one dimensional linear index
>     (a panning position) is all you really need.  Adding more
>     parameters will not improve the spatial rendering much.  Even if
>     you are not rendering with a simple linear array, creating spatial
>     audio with depth in telepresence rooms means placing the far-end
>     sound _behind_ the monitors - and adding rear speakers to the room
>     doesn't help you do that. I don't know any good way to render room
>     audio so it sounds like it is coming from an arbitrary depth
>     behind a wall-mounted monitor or projector screen.
>
>     However, if a middle box is selecting a subset of the audio
>     captures in a room, then it conceivably might be necessary to
>     provide more information about the pickup region.  So far I
>     haven't come up with good use case that shows that though.
>
[CNG] Whilst any indication to a user can be considered as "rendering", 
do you see that information about the pickup region could be used to 
generate some sort of local "service" indication i.e. for example 
highlighting part of a video region based on an voice activity of a 
certain audio capture.
>         <[email-masked]
>         <mailto:[email-masked]>
>         <mailto:[email-masked]
>         <mailto:[email-masked]>>> wrote:
>
>            Hello,
>
>         [email-masked] <mailto:[email-masked]> <mailto:[email-masked]
>         <mailto:[email-masked]>>
>

---

From: Christian Groves <[email-masked]>
Date: Fri, 02 Sep 2011 11:02:15 +1000

Hello Marshall

Please see my replies below.

Regards, Christian

On 2/09/2011 2:57 AM, Marshall Eubanks wrote:
> Dear Christian;
>
>
> On Thu, Sep 1, 2011 at 3:03 AM, Christian Groves 
>         - it is dangerous and limiting to attempt to capture equipment
>         spatial placement in a simple ordered list, especially when
>         that equipment (or its capture zone) can move and change with
>         time and that
>
>         - we are likely to be sending kilobytes of information about
>         codec formats and the like , so I don't see why we can't
>         afford a few bytes to
>         describe a mapping between equipment ids (numbering) and what
>         and where they are capturing.
>
>         There are three possible rankings of spatial equipment
>         capture, transverse, depth and vertical. Call them T, D and V.
>
>         Each can be declared to be relevant, with the assumption that
>         at least one will be. Ones not declared relevant are ignored.
>
>         Each can be declared to be fixed or variable width. In the
>         case of fixed width, the width is also declared.
>
>         Everything is referenced to the axis of symmetry of the unit,
>         oriented from the point of view of an observer standing on the
>         axis at the capture equipment facing the participants (i.e.,
>         the point of view of an outside observer). Order is right to
>         left (T), front to back (D) and bottom to top (V) . A
>         "starting" value is given (so that we don't have to deal with
>         negative numbers in the list) and a "unit" value (the smallest
>         addressable chunk of degrees or meters).
>
>     [CNG] Just to be 100% sure when you say axis of symmetry of "the
>     unit" do you mean each media capture device or the telepresence
>     system (TPS)? I guess when we start to take into geometries into
>     consideration the description (and implementation) does become
>     more complex. It might get interesting to consider a presentation
>     related capture.
>
>
>
> In many physics problems, determining the appropriate frame of 
> reference is half of the battle, and I suspect that this will be the 
> case here as well.
>
> videoconferencing and telepresence. There is, of course, a rather 
> longer history of running code in the theater and related arts. In the 
> theater, as in videoconferencing and telepresence, there is a local 
> audience. There are also performers (in theater) or remote 
> participants (in telepresence). Think about the most basic things in 
> that situation
>
> - There is a spatial division between the audience and the performers. 
> The audience is here, the performers are over there.
>
> right, up or down, but they almost never have to turn around and look 
> behind them. The performers occupy in the forward directions they can 
> look at comfortably.
>
> - There is almost  always a default direction (what the audience looks 
> at if they look straight ahead) which is an axis of symmetry for the 
> entire room. (That is true even for theater in the round.)
>
> This basic setup is seen across cultures, in theater, music, teachers 
> in schools, religious ceremonies, judges in courtrooms, etc. It was 
> familiar to the ancient Greeks, not to mention the Chinese, Japanese, 
> etc. It is also the basic setup for all of the telepresence solutions 
> I know of. There is so much running code here that if we can 
> accommodate it, we are probably "future-proof" for some time to come. 
> That is what I think we should aim for, and that is what I was trying 
> to do.
>
> So, I use the axis of symmetry as the basic reference point. Think of 
> it as, where the audience is looking. If someone can come up with a 
> better reference point, I am not wed to it, but I would want to 
> accommodate all of the above cases.

[CNG] OK so the intention was that the axis of symmetry is not really 
related to a device but related to the "scene"

> ...
>
>
>
>         I don't see this as really much more complicated or byte
>         consuming and it really would avoid a bunch of problems I see
>         as coming from overlaying a potentially variable 3-D spatial
>         order onto a simple list.
>
>     [CNG] Perhaps another approach is? to treat each capture set as
>     having "a spatial relationship" between media captures however
>     there is no significance of the position of the media capture in
>     the capture set definition itself (i.e. remove all spatial order
>     from the list). The spatial details related to the capture are
>     given as attributes of each media capture, e.g. T, D, V or other
>     as appropriate.
>
>
>

[CNG] Something like:

"MediaCaptures"
IDENTITY = STRING
Attributes = STRING
IDENTITY:Attributes[";" *Attributes]
"CaptureSets"
"(" IDENTITY [";" *IDENTITY]

e.g.
MediaCaptures
VC0:purpose=main;t=[-90:-30]
VC1:purpose=main;t=[-30:30]
VC2:purpose=main;t=[30:60]
CaptureSets
(VC1,VC0,VC2) or (VC0,VC2,VC1) or (VC0,VC1,VC3)

It doesn't matter what the order of the identities are all that the 
capture set indicates is that they are related. The actual spatial 
information is defined in the definition of the capture.

---

From: Marshall Eubanks <[email-masked]>
Date: Thu, 1 Sep 2011 21:23:05 -0400

Dear Christian;

On Thu, Sep 1, 2011 at 9:02 PM, Christian Groves <
[email-masked]> wrote:

>> <snip>
>> So, I use the axis of symmetry as the basic reference point. Think of it
>> as, where the audience is looking. If someone can come up with a better
If by "device" you mean "camera or microphone or ...", then, yes.

If by "device" you mean "telepresence unit, then I would argue that the axis
of symmetry is inherent  in the telepresence unit.

Note that the axis of symmetry just establishes the local angular reference
(i.e., what "60 degrees to the right" means). You also need a central
point for distances.  As Stephan indicates, I think we should have a
coordinate system for both capture and display.

> It doesn't matter what the order of the identities are

that I think should be the case.

> all that the capture set indicates is that they are related. The actual
> spatial information is defined in the definition of the capture.
>
>
>
So, VC0 captures far right to middle right (60 deg), V1 the center (30 deg).
VC2 the left (30 deg), all non-overlapping ?

That seems fine to me (the format, if not the cameras).

We should have something flexible enough to represent people at podiums and
the like, which I think this does with no effort.

Regards
Marshall

---

From: Stephen Botzko <[email-masked]>
Date: Thu, 1 Sep 2011 21:24:59 -0400

Hi Christian-
In-line

On Thu, Sep 1, 2011 at 8:42 PM, Christian Groves <
[email-masked]> wrote:

>> [SCB]Yes.  That makes it easy to match the sound stage to the displays,
>> and also deals nicely with the situation where the captures are not evenly
>> placed in the room.
>>
> problems. e.g. (AC0[lin_ind=30],AC1[lin_ind=**90],AC2[lin_ind=60])?
>

> [SCB] that makes sense to me, though perhaps if you are looking to select a
> particular stream (perhaps the one closest to the center) it would be
> convenient if they were ordered.
>

> complementary ways of describing this.

[SCB] My guess is that the basic information on the capture areas etc will
be the same, though I agree that we should be open to the idea that some of
the description is specific to audio or video. Sound behaves differently
from light.

> highlighting part of a video region based on an voice activity of a certain
> audio capture.
>

[SCB]If you detect voice activity in an audio capture, you absolutely want
to find the video capture[s] that carry the talker's image.  So I would
agree that the capture areas need to provide enough information to do that.

>>        <mailto:Christian.Groves@**nteczone.com<[email-masked]>
>> >
>>        <mailto:Christian.Groves@**nteczone.com<[email-masked]>
>>        <mailto:Christian.Groves@**nteczone.com<[email-masked]>>>>
>> wrote:
>>
>>           Hello,
>>

---

From: Christian Groves <[email-masked]>
Date: Fri, 02 Sep 2011 12:19:12 +1000

Hello Marshall,

On 2/09/2011 11:23 AM, Marshall Eubanks wrote:
> Dear Christian;
>
> On Thu, Sep 1, 2011 at 9:02 PM, Christian Groves 
>         So, I use the axis of symmetry as the basic reference point.
>         Think of it as, where the audience is looking. If someone can
>         come up with a better reference point, I am not wed to it, but
>         I would want to accommodate all of the above cases.
>
>
>     [CNG] OK so the intention was that the axis of symmetry is not
>     really related to a device but related to the "scene"
>
>
> If by "device" you mean "telepresence unit, then I would argue that 
> the axis of symmetry is inherent  in the telepresence unit.
>
> Note that the axis of symmetry just establishes the local angular 
> reference (i.e., what "60 degrees to the right" means). You also need 
> a central
>     all that the capture set indicates is that they are related. The
>     actual spatial information is defined in the definition of the
>     capture.
>
>
>
> So, VC0 captures far right to middle right (60 deg), V1 the center (30 
> deg). VC2 the left (30 deg), all non-overlapping ?
[CNG] Yes that's correct.

> That seems fine to me (the format, if not the cameras).
[CNG] At this stage I'm not proposing a particular format more just the 
principle that the spatial information is an attribute of the individual 
capture rather the capture set.
The T=<start_angle>,<end_angle> was to illustrate a spatial attribute.

I don't know whether to satisfy the minimum complexity camp whether we 
would need to compliment the geometric information with a further 
"hint". Say something like VC1:purpose=main;t=[-30:30.center] . I read 
<draft-polk-mmusic-traffic-class-for-sdp> with their structured approach 
where you can have an adjective and thought perhaps we could do 
something like this in CLUE for parameters where most cases a generic 
description will do but in other cases additional information may be 
beneficial.

>
> We should have something flexible enough to represent people at 
> podiums and the like, which I think this does with no effort.
>
> Regards
> Marshall
>
>
>
>
>         Regards
>         Marshall
>
>
>
>
>                Regards
>                Marshall
>
>                   Regards, Christian
>
>         <mailto:[email-masked]>> <mailto:[email-masked]
>         <mailto:[email-masked]>
>
>         <mailto:[email-masked] <mailto:[email-masked]>>>
>

---

From: Marshall Eubanks <[email-masked]>
Date: Thu, 1 Sep 2011 23:29:38 -0400

On Thu, Sep 1, 2011 at 10:19 PM, Christian Groves <
[email-masked]> wrote:

>> If by "device" you mean "telepresence unit, then I would argue that the
>> axis of symmetry is inherent  in the telepresence unit.
>>
>> reference (i.e., what "60 degrees to the right" means). You also need a
>> central
> would need to compliment the geometric information with a further "hint".
> Say something like VC1:purpose=main;t=[-30:30.**center]

What is your intent with "purpose=main" ? What are the options with that
field ?

> . I read <draft-polk-mmusic-traffic-**class-for-sdp> with their structured
> approach where you can have an adjective and thought perhaps we could do
Yes, that would be good.

Regards
Marshall

>
>
>> We should have something flexible enough to represent people at podiums
>> and the like, which I think this does with no effort.
>>
>> Regards
>> Marshall
>>
>>
>>
>>
>>        Regards
>>        Marshall
>>
>>
>>
>>
>>               Regards
>>               Marshall
>>
>>                  Regards, Christian
>>

---

From: Christian Groves <[email-masked]>
Date: Fri, 02 Sep 2011 18:50:25 +1000

Hello Marshall,

In-line below.

Regards, Christian

..snip..
>     [CNG] At this stage I'm not proposing a particular format more
>     just the principle that the spatial information is an attribute of
>     the individual capture rather the capture set.
>     I don't know whether to satisfy the minimum complexity camp
>     whether we would need to compliment the geometric information with
>     a further "hint". Say something like
>     VC1:purpose=main;t=[-30:30.center] 
>
>
> What is your intent with "purpose=main" ? What are the options with 
> that field ?

[CNG] This was just a copy of an attribute from the existing framework 
to show that there may be multiple attributes.

>     . I read <draft-polk-mmusic-traffic-class-for-sdp> with their
>     structured approach where you can have an adjective and thought
>     perhaps we could do something like this in CLUE for parameters
>     where most cases a generic description will do but in other cases
>     additional information may be beneficial.
>
>
> Yes, that would be good.
>
> Regards
> Marshall
>
>
>
>         We should have something flexible enough to represent people
>         at podiums and the like, which I think this does with no effort.
>
>         Regards
>         Marshall
>
>
>
>
>                Regards
>                Marshall
>
>
>
>
>                       Regards
>                       Marshall
>
>                          Regards, Christian
>
>         <mailto:[email-masked] <mailto:[email-masked]>
>         <mailto:[email-masked] <mailto:[email-masked]>>
>
>         <mailto:[email-masked] <mailto:[email-masked]>
>         <mailto:[email-masked] <mailto:[email-masked]>>>>
>
>

---

From: Marshall Eubanks <[email-masked]>
Date: Fri, 2 Sep 2011 07:01:12 -0400

On Fri, Sep 2, 2011 at 4:50 AM, Christian Groves <
[email-masked]> wrote:

>>    VC1:purpose=main;t=[-30:30.**center]
>>
> [CNG] This was just a copy of an attribute from the existing framework to
> show that there may be multiple attributes.
>
>
ACK,

Marshall

>
>     . I read <draft-polk-mmusic-traffic-**class-for-sdp> with their