Subject: [clue] Rendering mono sources using multiple loudspeakers

From: John Leslie <[email-masked]>
Date: Mon, 12 May 2014 10:02:40 -0400

Receiver tactics for rendering mono sources using multiple loudspeakers
====

   When individual microphone sources are available to a receiver,
the receiver may render the sound differently from the sender's mixer
output. This is not a trivial thing. But there will be cases where
the receiver must do this.

   A common case will be a remote individual (in an airport, perhaps)
participating in discussion of a single issue. That individual will
likely be shown in a more-or-less fixed position in the rendered
video; and we should trust the receiving sound manager to render the
mono audio to appear to come from that area of the screen (or be
muted when the signal/noise ration is too low).

   There will be other cases where, for whatever reason, there is no
usable full-room mix or there is no screen real-estate to show that
room; but individuals in that room speak, and the receiver can show
video including the person speaking. The interesting issue arises
when there are multiple microphone mediaCaptures and one must be
chosen to be rendered to match the video position.

   Clearly there are proprietary tricks outside the scope of this
standard. But we note that measuring relative delay of each available
microphone can show which microphone is closest to the speaker.
Unfortunately, this is difficult for the receiver -- while it is
easy for the sender to measure this.

   I recommend we consider a per-participant attribute to link to
the "nearest" microphone. In some cases this will be a lavalier
mike; in other cases it will be a fixed-position mike which the
sender calculates to be closest (whether determined statically or
dynamically). I think it would also help to include an attribute for
whether the microphone is worn (thus moving when the participant
moves) or attached to furniture, etc.

   Note that having a lavalier or gooseneck mike for each individual
expected to talk will be very helpful to some receivers.

--
John Leslie <[email-masked]>

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 12 May 2014 15:31:45 -0400

> muted when the signal/noise ration is too low).

The consumer is supposed to distinguish the above case from the below 
case by the presence of a single video and a single audio capture?

> standard.

Proprietary tricks that work with the information made available from an 
arbitrary clue peer? Or based on knowledge that both parties have some 
agreement in addition to clue? Such an agreement had better not require 
non-standard usage of the standard clue signaling and protocol. (Though 
it could be based on *extensions* to that.)

> But we note that measuring relative delay of each available
> the "nearest" microphone.

Do you really mean *participant*? Or do you mean per-video-capture?

If you really mean participant, how do you expect that to be used here?

	Thanks,
	Paul

> In some cases this will be a lavalier
> _______________________________________________
> clue mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/clue
>

---

From: John Leslie <[email-masked]>
Date: Mon, 12 May 2014 20:53:31 -0400

> case by the presence of a single video and a single audio capture?

   I really wasn't thinking in terms of "distinguishing two cases"...

> it could be based on *extensions* to that.)

   To me, "proprietary" mean "I'm not going to explain it!"
So, I am not expressing an opinion on what information they use...

>> easy for the sender to measure this.

   (Though I noted there that both sender and receiver _can_ use
this trick...)

> Do you really mean *participant*? Or do you mean per-video-capture?

   I was thinking per-participant when I wrote that.

> If you really mean participant, how do you expect that to be used here?

   I don't know: I don't believe we've settled what the participant
data will look ilke. (I could _guess_ we might have position information
available...)