Subject: [clue] Active speaker a role?

From: Christer Holmberg <[email-masked]>
Date: Thu, 7 Mar 2013 08:19:10 +0000

Hi,

A couple of questions related to the capture attribute draft.

Q1.

Assume you have a MCU, which collects streams from multiple endpoints, and provides the active speaker to a single screen entity. So, the content will change as the most active changes, but the capture provided by the MCU will be the same.

So, the "role" of this capture would be "active speaker". Or?

Q2.

I assume that capture attribute can also be used during the advertisement phase. Ie a single entity can indicate that it is able to receive a stream, where it want the content to be the "active speaker" role. Or?

Regards,

Christer

---

From: "Roni Even" <[email-masked]>
Date: Thu, 7 Mar 2013 10:54:13 +0200

Hi Christer,

This is a point I made in the past that for switched capture it may be good
to know what is the policy, can be active speaker, time switch (round robin)
or other methods.

As for your second question advertisement is what can be sent, you probably
meant that the configure message should allow to select a specific switch
policy, I think that the configure can only ask for something that was
advertised

Roni 

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Christer Holmberg
Sent: 07 March, 2013 10:19 AM
To: [email-masked]
Subject: [clue] Active speaker a role?

Hi,

Q1.

Assume you have a MCU, which collects streams from multiple endpoints, and
provides the active speaker to a single screen entity. So, the content will
change as the most active changes, but the capture provided by the MCU will
be the same.

 

Q2.

I assume that capture attribute can also be used during the advertisement
phase. Ie a single entity can indicate that it is able to receive a stream,
where it want the content to be the "active speaker" role. Or?

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 7 Mar 2013 11:32:23 +0000

Hi,

>This is a point I made in the past that for switched capture it may be good to know what is the policy, can be active speaker, time switch (round robin) or other methods.

Correct. So, the question is whether that policy can be seen as a "role".

There could also be "2nd recent speaker", "3rd recent speaker" etc, but instead of defining a separate role for each of them, maybe we could have an attribute value, e.g. "recent-speaker=2".

This type of functionality is very important for devices with a limited number of streams, screens etc - it non-telepresence rooms.

>As for your second question advertisement is what can be sent, you probably meant that the configure message should allow to select a specific switch policy, I think that the configure can only ask for something that was advertised

Yes.

Regards,

Christer

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Christer Holmberg
Sent: 07 March, 2013 10:19 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [clue] Active speaker a role?

Hi,

Q1.

Q2.

Regards,

Christer

---

From: "Roni Even" <[email-masked]>
Date: Thu, 7 Mar 2013 14:14:21 +0200

Hi Christer,

I think that if we use CLUE to signal the active speaker, previous, Etc it
will require many messages since the speakers change. The information who is
speaking can be available from the CSRC or using the RTP header extension
level attributes. RFC6465 provides the use case you discuss. 

Roni

From: Christer Holmberg [mailto:[email-masked]] 
Sent: 07 March, 2013 1:32 PM
To: Roni Even; [email-masked]
Subject: RE: [clue] Active speaker a role?

Hi,

There could also be "2nd recent speaker", "3rd recent speaker" etc, but
instead of defining a separate role for each of them, maybe we could have an
attribute value, e.g. "recent-speaker=2".

This type of functionality is very important for devices with a limited
number of streams, screens etc - it non-telepresence rooms.

Yes.

Regards,

Christer

Hi,

Q1.

 

Q2.

 

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 7 Mar 2013 12:28:00 +0000

Hi,

>I think that if we use CLUE to signal the active speaker, previous, Etc it will require many messages since the speakers change.

I am not sure I understand. The idea is to have a MCU capture, and e.g. a single stream, and the MCU switches the content based on who is active. There is no need to re-negotiate anything. Or, what messages do you refer to?

>The information who is speaking can be available from the CSRC or using the RTP header extension level attributes. RFC6465 provides the use case you discuss.

I am not talking about WHO (as in whether it's Alice, Bob or Carol) is talking, but that whoever is talking is provided from the MCU capture.

Regards,

Christer

From: Christer Holmberg [mailto:[email-masked]]
Sent: 07 March, 2013 1:32 PM
To: Roni Even; [email-masked]<mailto:[email-masked]>
Subject: RE: [clue] Active speaker a role?

Hi,

Yes.

Regards,

Christer

Hi,

Q1.

Q2.

Regards,

Christer

---

From: "Roni Even" <[email-masked]>
Date: Thu, 7 Mar 2013 14:52:01 +0200

Hi,

I will try to understand. What you are saying that we will have, for
example, three media captures, "current speaker","recent-1" and "recent-2".
This will be used for MCU that does voice activation video switching but
sends more than one RTP stream to each receiver.

Roni

From: Christer Holmberg [mailto:[email-masked]] 
Sent: 07 March, 2013 2:28 PM
To: Roni Even; [email-masked]
Subject: RE: [clue] Active speaker a role?

Hi,

>I think that if we use CLUE to signal the active speaker, previous, Etc it
will require many messages since the speakers change.

I am not sure I understand. The idea is to have a MCU capture, and e.g. a
single stream, and the MCU switches the content based on who is active.
There is no need to re-negotiate anything. Or, what messages do you refer
to?

>The information who is speaking can be available from the CSRC or using the
RTP header extension level attributes. RFC6465 provides the use case you
discuss. 

I am not talking about WHO (as in whether it's Alice, Bob or Carol) is
talking, but that whoever is talking is provided from the MCU capture.

Regards,

Christer

Hi,

Yes.

Regards,

Christer

Hi,

Q1.

 

Q2.

 

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 7 Mar 2013 13:14:10 +0000

Hi,

The MCU will advertise, to me, that it can provide three media captures (as in your example). The MCU will then receive media from all other participants, do voice activation, and forward the media streams with the "current speaker", "recenve-1" etc to me.

How the other endpoints provide their media to the MCU, and what roles are used, is a separate story.

Regards,

Christer

From: Roni Even [mailto:[email-masked]]
Sent: 7. maaliskuuta 2013 14:52
To: Christer Holmberg; [email-masked]
Subject: RE: [clue] Active speaker a role?

Hi,
I will try to understand. What you are saying that we will have, for example, three media captures, "current speaker","recent-1" and "recent-2". This will be used for MCU that does voice activation video switching but sends more than one RTP stream to each receiver.
Roni

Hi,

Regards,

Christer

Hi,

Yes.

Regards,

Christer

Hi,

Q1.

Q2.

Regards,

Christer

---

From: "Espen Berger (espeberg)" <[email-masked]>
Date: Thu, 7 Mar 2013 18:32:47 +0000

I suggest we split the problem into two parts.

1)      Subscribe to virtual captures, allow to server to forward based on information available on the server. The most normal scenario is to forward based on audio activity.

2)      Setting policy for how the server should switch streams. Policies could be

a.       Active speaker (based on audio level)

b.      Round robin

A client could then request two virtual streams (with capture id 1 and 2) with policy 'active speaker', that should mean that the two loudest speaker should be in either capture id 1 or 2, to avoid having the capture id changing.

-Espen

From: [email-masked] [mailto:[email-masked]] On Behalf Of Christer Holmberg
Sent: 7. mars 2013 14:14
To: Roni Even; [email-masked]
Subject: Re: [clue] Active speaker a role?

Hi,

Regards,

Christer

From: Roni Even [mailto:[email-masked]]
Sent: 7. maaliskuuta 2013 14:52
To: Christer Holmberg; [email-masked]<mailto:[email-masked]>
Subject: RE: [clue] Active speaker a role?

Hi,

Hi,

Regards,

Christer

Hi,

Yes.

Regards,

Christer

Hi,

Q1.

Q2.

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 7 Mar 2013 19:48:08 +0000

Hi,

>A client could then request two virtual streams (with capture id 1 and 2) with policy ‘active speaker’, that should mean that the two loudest speaker should be in either capture id 1 or 2, to avoid having the capture id changing.

I don't think the the active speaker should change between virtual captures - I'd rather see one virtual capture with the "active speaker" role. 

Because, I might negotiate a HD quality stream for the capture associated with the active speaker, while I request thumbnail quality stream(s) for other participant(s).

Regards,

Christer

Hi,

The MCU will advertise, to me, that it can provide three media captures (as in your example). The MCU will then receive media from all other participants, do voice activation, and forward the media streams with the “current speaker”, “recenve-1” etc to me.

Regards,

Christer

Hi,
I will try to understand. What you are saying that we will have, for example, three media captures, “current speaker”,”recent-1” and “recent-2”. This will be used for MCU that does voice activation video switching but sends more than one RTP stream to each receiver.
Roni

Hi,

>The information who is speaking can be available from the CSRC or using the RTP header extension level attributes. RFC6465 provides the use case you discuss.

I am not talking about WHO (as in whether it’s Alice, Bob or Carol) is talking, but that whoever is talking is provided from the MCU capture.

Regards,

Christer

Hi,

>This is a point I made in the past that for switched capture it may be good to know what is the policy, can be active speaker, time switch (round robin) or other methods.

Correct. So, the question is whether that policy can be seen as a “role”.

There could also be “2nd recent speaker”, “3rd recent speaker” etc, but instead of defining a separate role for each of them, maybe we could have an attribute value, e.g. “recent-speaker=2”.

This type of functionality is very important for devices with a limited number of streams, screens etc – it non-telepresence rooms.

Yes.

Regards,

Christer

Hi,

Q1.

So, the “role” of this capture would be “active speaker”. Or?

Q2.

I assume that capture attribute can also be used during the advertisement phase. Ie a single entity can indicate that it is able to receive a stream, where it want the content to be the “active speaker” role. Or?

Regards,

Christer

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 08 Mar 2013 04:04:32 +0800

This is an example of why we need a discussion of switched captures in 
the framework, including the difference between site switching and 
segment switching, and the relationship between switched video captures 
and switched audio captures.

	Thanks,
	Paul

On 3/8/13 3:48 AM, Christer Holmberg wrote:
> _______________________________________________
> clue mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/clue
>

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 7 Mar 2013 20:08:29 +0000

Hi,

>and switched audio captures.

The relationship between audio and video captures is a general issue, not only related to switching.

Regards,

Christer

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Mon, 11 Mar 2013 00:21:26 +0000

> -----Original Message-----
> Sent: Thursday, March 07, 2013 3:32 AM
> instead of defining a separate role for each of them, maybe we could have
> an attribute value, e.g. "recent-speaker=2".

How about addressing this using something similar to the multiple-render attribute proposed in:
http://tools.ietf.org/id/draft-jennings-rtcweb-plan-01.txt

In this case, you would have the role of active speaker and multiple-render=3, or however many active speakers can be sent/received.

Cheers,
Charles
 
> >As for your second question advertisement is what can be sent, you
> probably meant that the configure message should allow to select a specific
> switch policy, I think that the configure can only ask for something that was
> advertised
> 
> 
> 
> Yes.
> 
> 
> 
> Regards,
> 
> 
> 
> Christer
> 
> 
> 
> 
> 
> 
> 
> Assume you have a MCU, which collects streams from multiple endpoints,
> and provides the active speaker to a single screen entity. So, the content
> will change as the most active changes, but the capture provided by the
> MCU will be the same.
> 
> 
>

---

From: Qin Wu <[email-masked]>
Date: Tue, 12 Mar 2013 11:40:41 +0000

-----邮件原件-----
发件人: [email-masked] [mailto:[email-masked]] 代表 Charles Eckel (eckelcu)
发送时间: 2013年3月10日 20:21
收件人: Christer Holmberg; Roni Even; [email-masked]
主题: Re: [clue] Active speaker a role?

[Qin]: Sorry to chime in, it is not clear to me why the attribute multiple-render is really needed? Is this attributed used to deal with simulcast case?
Is "multiple streams are sent at the same time" same as "multiple streams are rendered at the same time"?
Why " max-send-ssrc" and " max-recv-ssrc" can not be used for this purpose?
Please correct me if I miss something.

Cheers,
Charles

---

From: Christian Groves <[email-masked]>
Date: Wed, 13 Mar 2013 17:46:48 +1100

Hello

Is it really needed to indicate via the CLUE signalling channel the "2nd
recent speaker", "3rd recent speaker" etc? It seems that this
information could be utilised by endpoints without CLUE. I think it
makes sense to have role information in CLUE as a means to select
captures to establish media streams for but utilising CLUE for keeping
lists of past speakers?

Regards, Christian

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 13 Mar 2013 15:06:21 +0000

Hi,

It is not about keeping a list.

The MCU is providing media streams representing the previous speakers. In a typical use-case they are shown as thumbnails.

Regards,

Christer

Sent from Windows using TouchDown (www.nitrodesk.com)

-----Original Message-----
From: Christian Groves [[email-masked]]
To: [email-masked] [[email-masked]]
Subject: Re: [clue] 答复: Active speaker a role?
Hello

Regards, Christian