Subject: [clue] Add floor-area intended-capture area for mikes and mixers

From: John Leslie <[email-masked]>
Date: Fri, 9 May 2014 10:43:52 -0400

Add a way to specify floor-area intended-capture area for mikes and mixers
====
   Currently we have spatialInformation within mediaCaptureType, and
spatialInformation includes captureArea.

   This has already led to much confusion. I suggest renaming
"captureArea" to "videoCaptureFrame" and adding "audioCapturePolygon"
to spatialInformation.

   The current video captureArea defines a trapezoid in a plane
more-or-less perpendicular to the floor. Actually the video includes
infinitely many rectangles at differing distances from the point of
capture, with no way to distinguish them except that some are out of
focus. Thus video can be thought of as insensitive to distance from
the camera.

   Audio mediaCaptures, however, are mostly insensitive to height above
the floor, but necessarily concentrate on capturing the sound within
some distance of the microphone. The audio level decreases with the
square of the distance.

   While microphones pick up _some_ sound from anywhere in the room,
the _intent_ in setting up microphones is to pick up sound in a
defined area. This deserves a way to specify it -- at least in the case
where that microphone's sound output is separately available.

   In many cases, the sound engineer will wish to only offer one mixer
output (hopefully stereo or surround) for the entire room. But when the
individual microphones are separately available, it would be very helpful
to know the area each is intended to cover. This will not be a rectangle,
so I advise defining it as a polygon.

--
John Leslie <[email-masked]>

---

From: Christian Groves <[email-masked]>
Date: Mon, 12 May 2014 13:03:59 +1000

Hello John,

Do you propose that the audioCapturePolygon has a relationship to the 
capturePoint/lineOfCapturePoint.

Regards, Christian

On 10/05/2014 12:43 AM, John Leslie wrote:
> _______________________________________________
> clue mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/clue
>

---

From: John Leslie <[email-masked]>
Date: Mon, 12 May 2014 07:17:53 -0400

> capturePoint/lineOfCapturePoint.

   There is (IMHO) no necessary relationship. Sometimes the microphone
will be a boundary microphone positioned on a table (with axis upward);
sometimes it will be a mix of cardioid microphones aimed inward; and
there are _many_ other possibilities.

---

From: Christian Groves <[email-masked]>
Date: Mon, 12 May 2014 21:47:41 +1000

OK then we should make it clear if we go for audioCapturePolygon that 
the capturePoint/lineOfCapturePoint only apply to the video case.

Christian

---

From: John Leslie <[email-masked]>
Date: Mon, 12 May 2014 08:01:53 -0400

> the capturePoint/lineOfCapturePoint only apply to the video case.

   I don't follow... audioCapturePolygon is about the _intent_, while
capturePoint is about what the microphone _is_ recording. These are
different purposes.

   Note, BTW, that a mixer output _already_ has no capturePoint. Both
metrics are necessarily optional...

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 12 May 2014 11:05:09 -0400

> capturePoint is about what the microphone _is_ recording. These are

I think I understand, to some degree. But you said that the audio level 
drops with the square of the distance from the mic. So is the capture 
polygon to be evaluated together with the point of capture?

Is it your intent that the audio capture polygon be meaningful for 
mixes, while the point of capture isn't?

	Thanks,
	Paul

---

From: John Leslie <[email-masked]>
Date: Mon, 12 May 2014 11:24:34 -0400

Paul Kyzivat <[email-masked]> wrote:
> drops with the square of the distance from the mic.

   Always, yes.

> So is the capture polygon to be evaluated together with the point
> of capture?

   I see no reason to do so.

   If a capture-polygon is given, the intent is that the entire
contained area is "uniform enough".

   A receiver _could_, for an audioCapture with both axis-of-capture
and capture-polygon, evaluate distance-to-speaker for participants;
but it's probably not worth one's while to do anything about it.

   Most of the time, IMHO, you won't get both -- so I definitely
wouldn't bother.

> mixes, while the point of capture isn't?

   It's my expectation that it won't be there at all in most cases.

   Point-of-capture is typically meaningless for a mix -- only
point-of-capture for the inputs to that mix make sense to me.

---

From: Paul Coverdale <[email-masked]>
Date: Mon, 12 May 2014 12:54:23 -0400

I'm still having a hard time trying to understand what capture area (or
polygon) means for audio. The sound waves that a microphone collects are
defined by a volume, not an area. The waves emanating from further away just
take longer to be picked up. So if you want to inform the other telepresence
end-point as to the details of your local microphone arrangement (which I
assume is what we're trying to do in CLUE....), it seems to me that the only
useful information is the physical location of each microphone, its generic
pattern type and its axis of orientation.

Of course, one could say the same thing about using capture area for video -
it implies zero depth of field.

...Paul

>-----Original Message-----
>From: clue [mailto:[email-masked]] On Behalf Of John Leslie
>Sent: Monday, May 12, 2014 11:25 AM
>To: Paul Kyzivat
>Cc: [email-masked]
>Subject: Re: [clue] Add floor-area intended-capture area for mikes and
>mixers
>
>>>>> will be a boundary microphone positioned on a table (with axis
>>>>> upward); sometimes it will be a mix of cardioid microphones aimed
>>>>> inward; and there are _many_ other possibilities.
>>>>
>>>> OK then we should make it clear if we go for audioCapturePolygon
>>>> that the capturePoint/lineOfCapturePoint only apply to the video
>case.
>>>
>> I think I understand, to some degree. But you said that the audio
>> level drops with the square of the distance from the mic.
>
>   Always, yes.
>
>> So is the capture polygon to be evaluated together with the point of
>> capture?
>
>   A receiver _could_, for an audioCapture with both axis-of-capture and
>capture-polygon, evaluate distance-to-speaker for participants; but it's
>probably not worth one's while to do anything about it.
>
>   Point-of-capture is typically meaningless for a mix -- only point-of-
>capture for the inputs to that mix make sense to me.
>
>--

---

From: John Leslie <[email-masked]>
Date: Mon, 12 May 2014 13:16:37 -0400

> pattern type and its axis of orientation.

   I mostly agree.

> it implies zero depth of field.

   (Which is nonsense!)

   It _is_ possible, of course, to derive the pyramid which is the actual
field of a camera; and it is possible to, for any point within that
pyramid, determine the "closest" microphone (adjusting for sensitivity
pattern).

   Folks don't seem to want to do that.

   So I explained a shortcut -- measuring propagation time to find the
actually nearest microphone. This gets dicey, IMHO, unless it's done
by the sender (which is the only party with all the possible sound
sources synchronized in time.

   Thus the sender could do that calculation and rank distances from
persons speaking to available microphones.

   Folks don't seem a lot more enthusiastic about that.

   Similarly, the sender could have the sound engineer record the
_intended_ coverage area of microphones: that is what I recommended
adding a way to record.

   Whether that will generate any enthusiasm, only time will tell.
But it is a reasonably easy thing for any sound engineer who cares
to do. Thus I thought it worth providing a way to signal it.

   YMMV, as always...

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 12 May 2014 13:49:50 -0400

> it implies zero depth of field.

This has always troubled me a bit. I have been assuming that there is 
really a volume of capture that consists of the pyramid with peak at the 
point of capture with edges passing through the boundary of the area of 
capture, and continuing to the (unspecified) boundary of the scene. And 
that the area of capture is expected to be in focus, while the rest 
might not be.

	Thanks,
	Paul

---

From: "Duckworth, Mark" <[email-masked]>
Date: Mon, 12 May 2014 10:55:26 -0700

Hi Paul,

I agree with your assumption, but it doesn't trouble me :)
And sometimes there won't be any point of capture, for example for a
composed capture, so there is no defined volume.  Yet the area of capture is still useful.

Mark

> From: clue [mailto:[email-masked]] On Behalf Of Paul Kyzivat
> Sent: Monday, May 12, 2014 1:50 PM
> To: Paul Coverdale; 'John Leslie'
> Cc: [email-masked]
> > polygon) means for audio. The sound waves that a microphone collects
> > are defined by a volume, not an area. The waves emanating from further
> > away just take longer to be picked up. So if you want to inform the
> > other telepresence end-point as to the details of your local
> > microphone arrangement (which I assume is what we're trying to do in
> > CLUE....), it seems to me that the only useful information is the
> > physical location of each microphone, its generic pattern type and its axis of
> orientation.
> >
> > Of course, one could say the same thing about using capture area for
> > video - it implies zero depth of field.
> 
> This has always troubled me a bit. I have been assuming that there is really a
> volume of capture that consists of the pyramid with peak at the point of
> capture with edges passing through the boundary of the area of capture, and
> continuing to the (unspecified) boundary of the scene. And that the area of
> capture is expected to be in focus, while the rest might not be.
> 
> 	Thanks,
> 	Paul
> 
> > ...Paul
> >
> >> Subject: Re: [clue] Add floor-area intended-capture area for mikes
> >> and mixers
> >>
> >>>>>> There is (IMHO) no necessary relationship. Sometimes the
> >>>>>> microphone will be a boundary microphone positioned on a table
> >>>>>> (with axis upward); sometimes it will be a mix of cardioid
> >>>>>> microphones aimed inward; and there are _many_ other
> possibilities.
> >>>>>
> >> point-of- capture for the inputs to that mix make sense to me.
> >>
> >> --

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 12 May 2014 14:16:17 -0400

> composed capture, so there is no defined volume.  Yet the area of capture is still useful.

Yes, there might not be a point of capture. In that case we need to take 
greater pains to specify what whe area of capture *means*.

If it is synthetic (e.g. a powerpoint) then this makes sense.
A composed capture can probably be treated the same way.

But then these cases present further problems for associating audio and 
video.

	Thanks,
	Paul