Subject: Re: [clue] Audio channel format - RE: Note that XY-stereo microphones are commonly available

From: "Duckworth, Mark" <[email-masked]>
Date: Mon, 16 Jun 2014 07:22:23 -0700

I don't think we resolved this issue, about the media capture attribute "audio channel format", and how it relates to the audio encoding parameters that are negotiated in SDP.  I have a revised suggestion to resolve it.  Does anybody else have input on this issue?

>From the May 13 2014 design team minutes:

    "It was agreed that spatial audio in general should be included with the codec and will be done in SDP."

But there were also other issues about different formats such as "separated mic stereo" and "XY stereo".  I think binaural would fit here too.  These are different 2-channel formats, that could all use the same codec and encoding parameters, but yet would still have different meaning at a higher level that could affect how the rendering side chooses streams to receive and how it renders.  I think a media capture attribute is the right place in the CLUE model to describe this.

So my revised suggestion is to keep the "audio channel format" capture attribute, but add some additional possible values:
- mono
- separated mic stereo
- XY stereo
- binaural
- 3-channel (left, center, right, like separated mic stereo but with 3 channels)
- 5.1 surround? (not sure where to draw the line on more multi-channel formats)

I think the framework should also add some better explanation of the difference between the *capture* format and the *encoding* format.  For example a provider could offer a 3-channel capture as a 3-channel or stereo or mono encoding (potentially with different codecs), to be compatible with receivers that can't receive 3-channel.  (Existing non-CLUE equipment does this today with different available encodings for the same capture, without specifying the audio channel format at the capture level).

Regards,
Mark

> -----Original Message-----
> From: John Leslie [mailto:[email-masked]]
> Sent: Monday, May 12, 2014 9:06 PM
> To: Duckworth, Mark
> Cc: John Leslie; [email-masked]
> Subject: Re: Audio channel format - RE: [clue] Note that XY-stereo
> microphones are commonly available
> 
> Duckworth, Mark <[email-masked]> wrote:
> > [John Leslie wrote:]
> >> Duckworth, Mark <[email-masked]> wrote:
> >>>
> >>> I'm thinking this "audio channel format" parameter is really a
> >>> characteristic of a particular audio *encoding*, not an audio
> >>> *capture*.
> >>
> >>> So my idea is we should remove this audio capture attribute from the
> >>> framework and data model. The format of a particular audio encoding
> >>> would be described entirely in SDP (not CLUE messages).
> >>
> >> I'm not _opposed_ to that; but there are _so_many_details_...
> 
>    Alas, the bottom-line, regarding tomorrows design-team call, is that I won't
> have time to get into details before mid-week. :^(
> 
> >>> And a media provider could advertise one single audio capture for
> >>> the whole room, which maybe it could encode as mono, or stereo, or
> >>> 3 channel, or surround sound, or whatever. Or maybe even offer
> >>> multiple encodings for simulcast, but these are still all just
> >>> different encodings of the same audio capture.
> 
>    (Note that there are other cases than whole-room capture that we should
> deal with, IMHO.)
> 
> >> I'm not sure how to manage that, though. If a mix is prepared in
> >> 5.1 surround, I doubt that SDP is the right conduit for negotiating
> >> the number of channels -- offhand that sounds like a middlebox task.
> >
> > [Duckworth, Mark] Maybe we're talking about different things, because
> > I think SDP is exactly the right thing for this, and is commonly done
> > today, at least with stereo and mono.
> 
>    No doubt it would help for Mark and me to exchange email as I study this
> later this week.
> 
> > Endpoints I am familiar with today are capable of encoding their
> > outgoing room audio as mono, stereo, and in some cases 3 channel.
> > If the receiving side (whether middlebox or another endpoint) is
> > capable only of mono, then the sender and receiver negotiate mono.
> > This is done with SDP offer/answer today. And I think it should be
> > done that way with CLUE also.
> 
>    That sounds superficially reasonable. Some details which come to mind are
> how to proceed when receiving audio from two or more sending rooms...
> 
> >> And I'm _sure_ SDP isn't the right conduit for remapping the
> >> components of a whole-room feed to fit an incompatible conference
> room paradigm.
> 
>    Well, I was sure yesterday, anyway. ;^)
> 
> > [Duckworth, Mark] It isn't about conserving bandwidth. It is about a
> > receiver selecting an audio encoding format it is capable of decoding.
> 
>    That does sound like an SDP issue.
> 
>    I had assumed multi-room teleconferencing systems would be able to
> decode at least one multi-channel surround format.
> 
> --
> John Leslie <[email-masked]>

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 16 Jun 2014 10:50:14 -0400

> - 5.1 surround? (not sure where to draw the line on more multi-channel formats)

This seems reasonable to me.

> I think the framework should also add some better explanation of the difference between the *capture* format and the *encoding* format.
> For example a provider could offer a 3-channel capture as a 3-channel or stereo or mono encoding (potentially with different codecs), to be compatible with receivers that can't receive 3-channel.
> (Existing non-CLUE equipment does this today with different available encodings for the same capture, without specifying the audio channel format at the capture level).

Hmm. Here you are talking about a single *capture* represented via 
different encodings. We have decided that encodings are represented 
solely in SDP. So the differences you are talking about here need to be 
representable in SDP. But above you are calling out mono and stereo as 
*capture* formats.

OK, lets talk about this in the design team call tomorrow.

We have no proposal close the "issue" on echo, so we are not going to 
talk about that tomorrow.

	Thanks,
	Paul

> _______________________________________________
> clue mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/clue
>

---

From: John Leslie <[email-masked]>
Date: Mon, 16 Jun 2014 11:06:53 -0400

Duckworth, Mark <[email-masked]> wrote:
> 
> I don't think we resolved this issue, about the media capture attribute
> "audio channel format", and how it relates to the audio encoding
> parameters that are negotiated in SDP.

   My memory is foggy...

> I have a revised suggestion to resolve it. Does anybody else have input
> on this issue?
> 

   That doesn't exactly fit my memory...

> But there were also other issues about different formats such as
> "separated mic stereo" and "XY stereo". I think binaural would fit here
> too. These are different 2-channel formats, that could all use the same
> codec and encoding parameters, but yet would still have different
> meaning at a higher level that could affect how the rendering side
> chooses streams to receive and how it renders.

   I quite agree.

> I think a media capture attribute is the right place in the CLUE model
> to describe this.
> 
> So my revised suggestion is to keep the "audio channel format" capture
> attribute, but add some additional possible values:
> - mono
> - separated mic stereo
> - XY stereo
> - binaural

   +1

   On the surround-sound question, IMHO 5.1 is the most common form, and
others can be processed to it easily enough.

> I think the framework should also add some better explanation of the
> difference between the *capture* format and the *encoding* format.

   +1

---

From: "Duckworth, Mark" <[email-masked]>
Date: Mon, 16 Jun 2014 08:08:16 -0700

Hi Paul,

I'm saying the *encoding* information needs to be represented in SDP, and I believe it already is with no additions needed for CLUE.  This encoding information about the codec and codec parameters includes information about the number of audio channels in the encoding.  But suppose it is a 2-channel encoding, it still does not say anything about which type of 2-channel encoding it is (separated mic, XY, binaural).  My suggestion is to leave that information as a capture attribute.  I suppose we would also need to specify what it means if a surround or 3-channel capture is sent as a 2-channel encoding - which 2-channel format is it?  Maybe we can say it should be treated as separated mic stereo?  Or maybe the capture attribute should allow multiple values so the provider can specify it?

So far, the only encoding format I know about that would benefit from the additional capture attribute information is a 2-channel encoding.  If the encoding is mono, 3-channel, or surround, then I don't have any example of what additional information the capture attribute could provide.

Mark

> From: clue [mailto:[email-masked]] On Behalf Of Paul Kyzivat
> Sent: Monday, June 16, 2014 10:50 AM
> To: [email-masked]
> Subject: Re: [clue] Audio channel format - RE: Note that XY-stereo
> "audio channel format", and how it relates to the audio encoding parameters
> that are negotiated in SDP.  I have a revised suggestion to resolve it.  Does
> anybody else have input on this issue?
> >
> "separated mic stereo" and "XY stereo".  I think binaural would fit here too.
> These are different 2-channel formats, that could all use the same codec and
> encoding parameters, but yet would still have different meaning at a higher
> level that could affect how the rendering side chooses streams to receive
> and how it renders.  I think a media capture attribute is the right place in the
> CLUE model to describe this.
> >
> > For example a provider could offer a 3-channel capture as a 3-channel or
> stereo or mono encoding (potentially with different codecs), to be
> compatible with receivers that can't receive 3-channel.
> > (Existing non-CLUE equipment does this today with different available
> encodings for the same capture, without specifying the audio channel format
> at the capture level).
> 
> different encodings. We have decided that encodings are represented solely
> in SDP. So the differences you are talking about here need to be
> We have no proposal close the "issue" on echo, so we are not going to talk
> about that tomorrow.
> 
> 	Thanks,
> 	Paul
> 
> > Regards,
> > Mark
> >
> >>>>> So my idea is we should remove this audio capture attribute from
> >>>>> the framework and data model. The format of a particular audio
> >>>>> encoding would be described entirely in SDP (not CLUE messages).
> >>>>
> >>     Alas, the bottom-line, regarding tomorrows design-team call, is
> >> that I won't have time to get into details before mid-week. :^(
> >>
> >>     (Note that there are other cases than whole-room capture that we
> >> should deal with, IMHO.)
> >>
> >>> [Duckworth, Mark] Maybe we're talking about different things,
> >>> because I think SDP is exactly the right thing for this, and is
> >>> commonly done today, at least with stereo and mono.
> >>
> >>     No doubt it would help for Mark and me to exchange email as I
> >> study this later this week.
> >>
> >>     That sounds superficially reasonable. Some details which come to
> >> mind are how to proceed when receiving audio from two or more sending
> rooms...
> >>
> >>     I had assumed multi-room teleconferencing systems would be able
> >> to decode at least one multi-channel surround format.
> >>
> >> --

---

From: John Leslie <[email-masked]>
Date: Mon, 16 Jun 2014 15:05:54 -0400

> talk about that tomorrow.

   It's OK with me to not discuss it; but I ought to bring folks up to date.

>From the minutes I found at

http://trac.tools.ietf.org/wg/clue/trac/attachment/wiki/Design-Team/minutes-140513.txt

] 8. Whether echo cancellation is done
] This issue provoked a lot of discussion. Paul C has the view that, while
] echo cancellation is certainly important for Telepresence, it is beyond
] the scope of CLUE. John L's view is that it is too important to be left
] to "chance", and something explicit needs to be signaled. Paul K will
] issue a call for consensus on the mailing list.

   I've done some substantial research on this; and I've found that my
proposal on the 13th was simply wrong.

   There really is no single definition of what "echo cancellation" means
in terms of echo reduction decibels, and there's no reason to expect one
to magically appear. The only definitons of "echo suppression" in terms
of decibels amount to "drop the power to zero and add comfort noise" --
which is even less useful.

   I have since found definitions of "Echo Return Loss" and "Echo Return
Loss Enhancement" (in Cisco documentation as well as several other
vendors) which strike me as useful.

   "Echo Return Loss" is the number of decibels that the echo picked up
at the microphone is below the received signal sent to the loudspeaker(s).
"Echo Return Loss Enhancement" is the number of decibels below that which
is accomplished by echo-cancellation.

   These are already-defined numbers which IMHO would be useful to pass
from sending sites to receiving sites. The receiving sites could decline
to use audio feeds where these numbers are too low; or they could enforce
echo-suppression to give a single-duplex experience to sites with
inadequate echo-management numbers.

   It remains (IMHO) far better to know the problem in advance.

   (BTW, the IESG has given up on calling the roll to see who's present;
so the level of constant annoyance I've been experiencing has dropped. ;^)