Subject: [clue] CLUE and Simulcast - ticket #47

From: "Roni Even" <[email-masked]>
Date: Wed, 22 Oct 2014 00:00:48 +0300

Hi,

During our weekly design team meeting we discussed simulcast support in
CLUE.  

Here is my take on the options, my description may not be accurate and the
purpose here is to continue the discussion allowing us to have a better view
of the options.

Looking at the state of simulcast support in other WGs, there is an
individual draft
http://tools.ietf.org/html/draft-westerlund-avtcore-rtp-simulcast-04 that
being discussed. The proposed solution is based on the unified plan where
all encoded streams composing alternatives of a simulcast stream are in the
same m-line.   The alternatives are described using an SDP media level
attribute "a=simulcast". This attribute is not defined for SDP session
level.

The first question we had is if we should base CLUE simulcast support on
this individual draft based on CLUE timeline or try to define a different
multicast solution at the CLUE protocol level.

If we base our solution on the avtcore-rtp-simulcast-04 document the
advertisement will include an individual encoding that will relate to
optional multiple simulcast encoding and the configure will select such
encoding. The actual number of different stream that will compose the
simulcast stream will be negotiated using Offer/Answer as described in the
above document. The risk here is that this is an individual draft and we may
need to review the CLUE support if it changes. The advantage is that we will
have simulcast the same for CLUE and non-CLUE systems.

The other option for simulcast support in the CLUE protocol can be based on
having separate Media Capture for each of the different simulcast encoding
and use MCC to compose the simulcast alternatives. We will need some way to
escribe that this MCC is for simulcast description and to provide the actual
allowed individual encodes combinations for the Configuration by the
consumer.

Thanks

Roni

---

From: Christian Groves <[email-masked]>
Date: Fri, 24 Oct 2014 12:58:35 +1100

Hello Roni,

Supporting draft-westerlund-avtcore-rtp-simulcast-04 
<http://tools.ietf.org/html/draft-westerlund-avtcore-rtp-simulcast-04> 
seems like the most straight forward approach as it keeps things at the 
encoding level, which is really where simulcast belongs.

For a CLUE based solution I don't think we need to revert to MCCs. The 
purpose of simulcast is to provide different encoding options for a 
particular media stream. A media provider can Advertise a MC with 
multiple encoding groups. A media consumer if it wants to receive 
multiple encodings for the same MC can simply choose multiple capture 
encodings e.g. MC1/Encoding1 MC1/Encoding2 etc.

Regards, Christian

On 22/10/2014 8:00 AM, Roni Even wrote:
>
> Hi,
>
> During our weekly design team meeting we discussed simulcast support 
> in CLUE.
>
> Here is my take on the options, my description may not be accurate and 
> the purpose here is to continue the discussion allowing us to have a 
> better view of the options.
>
> http://tools.ietf.org/html/draft-westerlund-avtcore-rtp-simulcast-04 
> that being discussed. The proposed solution is based on the unified 
> plan where all encoded streams composing alternatives of a simulcast 
> stream are in the same m-line. The alternatives are described using an 
> SDP media level attribute “a=simulcast”. This attribute is not defined 
> for SDP session level.
>
> The first question we had is if we should base CLUE simulcast support 
> on this individual draft based on CLUE timeline or try to define a 
> different multicast solution at the CLUE protocol level.
>
> optional multiple simulcast encoding and the configure will select 
> such encoding. The actual number of different stream that will compose 
> the simulcast stream will be negotiated using Offer/Answer as 
> described in the above document. The risk here is that this is an 
> individual draft and we may need to review the CLUE support if it 
> changes. The advantage is that we will have simulcast the same for 
> CLUE and non-CLUE systems.
>
> The other option for simulcast support in the CLUE protocol can be 
> based on having separate Media Capture for each of the different 
> simulcast encoding and use MCC to compose the simulcast alternatives. 
> We will need some way to escribe that this MCC is for simulcast 
> description and to provide the actual allowed individual encodes 
> combinations for the Configuration by the consumer.
>
> Thanks
>
> Roni
>
>
>
> _______________________________________________
> clue mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/clue

---

From: "Roni Even" <[email-masked]>
Date: Fri, 24 Oct 2014 15:18:53 +0300

Hi Christian,
Inline
Roni

> -----Original Message-----
> From: clue [mailto:[email-masked]] On Behalf Of Christian Groves
> Sent: 24 October, 2014 4:59 AM
> To: [email-masked]
> Subject: Re: [clue] CLUE and Simulcast - ticket #47
> 
> Hello Roni,
> 
purpose
> of simulcast is to provide different encoding options for a particular
media
> stream. A media provider can Advertise a MC with multiple encoding groups.
A
> media consumer if it wants to receive multiple encodings for the same MC
can
> simply choose multiple capture encodings e.g. MC1/Encoding1 MC1/Encoding2
> etc.
[Roni Even] How does the media consumer know that these are alternatives of
a simulcast stream and know how many can be used at the same time. You also
lose here the option of the total BW that is an encoding group attribute.
All these means that we need some changes to CLUE and we can look at what
will make the best solution using MCs or MMCs.
Still the question is if we say that we will use the SDP based solution even
though it is not a WG draft, having it as a normative reference will delay
the publication of the CLUE draft

> > SDP media level attribute "a=simulcast". This attribute is not defined
> > for SDP session level.
> >

---

From: "Duckworth, Mark" <[email-masked]>
Date: Fri, 24 Oct 2014 06:47:34 -0700

Hello Roni and Christian,

I believe CLUE already supports simulcast today, basically as Christian describes.

More below.

(trying to use both the "correct indentation with > > symbols", and also marking
comments with my name :-)

Mark

> From: clue [mailto:[email-masked]] On Behalf Of Roni Even
> Sent: Friday, October 24, 2014 8:19 AM
> To: 'Christian Groves'; [email-masked]

[Duckworth, Mark] Does CLUE really need to say anything about this?
Isn't this separate from CLUE, and can be used with CLUE without any further
specification in CLUE documents?

> > For a CLUE based solution I don't think we need to revert to MCCs. The purpose
> > of simulcast is to provide different encoding options for a particular media
> > A media consumer if it wants to receive multiple encodings for the same MC
> > can simply choose multiple capture encodings e.g. MC1/Encoding1
> > MC1/Encoding2 etc.

[Duckworth, Mark] Yes, a media provider can support multiple encodings for the
same MC, and a consumer can choose to configure multiple encodings for the 
same MC.  You don't even need multiple encoding groups to do this, you can do 
it with just one group that contains multiple encodings.

> a simulcast stream and know how many can be used at the same time.

[Duckworth, Mark] By the MC attribute "max capture encodings".  Any encodings
in the MC's encoding groups can be configured for a particular MC, simultaneously,
up to the value of "max capture encodings".

>You also lose here the option of the total BW that is an encoding group attribute.

[Duckworth, Mark] You can still use that by advertising an MC with a single encoding
group, with multiple encodings in the group.

> will make the best solution using MCs or MMCs.

[Duckworth, Mark] I believe we don't need any changes in CLUE, all this is 
already supported.

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 24 Oct 2014 12:57:15 -0400

I agree with Mark here.

Consider a plausible use case in clue:

The advertiser has a number of captures, and has a number of encoders. 
It can create an encoding group with all the encoders in it - some do 
hi-res video, some do small videos. It puts all of those in an encoding 
group and associates all the captures with that group.

A "normal" consumer might configure a few of the captures with the 
hi-res encodings, and some of the other captures with the small 
encodings. There would be no simulcast in this case.

Some other consumer (e.g., an MCC) may decide it wants one of the 
captures in both big and small versions of one of the MCCs. It can 
simply do an appropriate configuration and get that. Now we have 
*simulcast*! The advertiser didn't have to do anything special - didn't 
have to think about simulcast at all.

If an advertiser *wants* to use draft-westerlund-avtcore-rtp-simulcast 
then I see no reason to prevent that. But I would hate to force that 
(especially since it isn't approved yet).

	Thanks,
	Paul

---

From: "Roni Even" <[email-masked]>
Date: Sat, 25 Oct 2014 17:53:34 +0300

Hi Paul,
Inline
Roni

> From: clue [mailto:[email-masked]] On Behalf Of Paul Kyzivat
> Sent: 24 October, 2014 7:57 PM
> To: [email-masked]
hi-res
> video, some do small videos. It puts all of those in an encoding group and
> associates all the captures with that group.
> 
> A "normal" consumer might configure a few of the captures with the hi-res
> encodings, and some of the other captures with the small encodings. There
> would be no simulcast in this case.
> 
> Some other consumer (e.g., an MCC) may decide it wants one of the captures
> in both big and small versions of one of the MCCs. It can simply do an
> appropriate configuration and get that. Now we have *simulcast*! The
> advertiser didn't have to do anything special - didn't have to think about
> simulcast at all.
[Roni Even] How does the consumer know that these MCs are different
potential encodings of the same source. 
Marks, suggested different encoding group for each encoding so he can
advertise MC1 with ENC1 and ENC2, in this case the encoding groups will
represent different encoding parameters but will need to be defined
accordingly, for example one encoding group that has CIF resolutions and the
other with only HD resolutions

> then I see no reason to prevent that. But I would hate to force that
(especially
> since it isn't approved yet).
> 
> 	Thanks,
> 	Paul
> 
> > (trying to use both the "correct indentation with > > symbols", and
> > also marking comments with my name :-)
> >
> > Mark
> >
> >>> <http://tools.ietf.org/html/draft-westerlund-avtcore-rtp-simulcast-0
> >>> 4> seems like the most straight forward approach as it keeps things
> >>> at the encoding level, which is really where simulcast belongs.
> >
> > Isn't this separate from CLUE, and can be used with CLUE without any
> > further specification in CLUE documents?
> >
> >>> For a CLUE based solution I don't think we need to revert to MCCs.
> >>> The purpose of simulcast is to provide different encoding options
> >>> for a particular media stream. A media provider can Advertise a MC
with
> multiple encoding groups.
> >>> A media consumer if it wants to receive multiple encodings for the
> >>> same MC can simply choose multiple capture encodings e.g.
> >>> MC1/Encoding1
> >>> MC1/Encoding2 etc.
> >
> > [Duckworth, Mark] Yes, a media provider can support multiple encodings
> > for the same MC, and a consumer can choose to configure multiple
> > encodings for the same MC.  You don't even need multiple encoding
> > groups to do this, you can do it with just one group that contains
multiple
> encodings.
> >
> >> [Roni Even] How does the media consumer know that these are
> >> alternatives of a simulcast stream and know how many can be used at the
> same time.
> >
> > [Duckworth, Mark] By the MC attribute "max capture encodings".  Any
> > encodings in the MC's encoding groups can be configured for a
> > particular MC, simultaneously, up to the value of "max capture
encodings".
> > [Duckworth, Mark] You can still use that by advertising an MC with a
> > single encoding group, with multiple encodings in the group.
> >
> >> All these means that we need some changes to CLUE and we can look at
> >> what will make the best solution using MCs or MMCs.
> >
> > [Duckworth, Mark] I believe we don't need any changes in CLUE, all
> > this is already supported.
> >
> >> Still the question is if we say that we will use the SDP based
> >> solution even though it is not a WG draft, having it as a normative
> >> reference will delay the publication of the CLUE draft
> >>
> >>>
> >>> Regards, Christian
> >>>
> >>>> During our weekly design team meeting we discussed simulcast
> >>>> support in CLUE.
> >>>>
> >>>> Here is my take on the options, my description may not be accurate
> >>>> and the purpose here is to continue the discussion allowing us to
> >>>> have a better view of the options.
> >>>>
> >>>> http://tools.ietf.org/html/draft-westerlund-avtcore-rtp-simulcast-0
> >>>> 4 that being discussed. The proposed solution is based on the
> >>>> unified plan where all encoded streams composing alternatives of a
> >>>> simulcast stream are in the same m-line. The alternatives are
> >>>> described using an SDP media level attribute "a=simulcast". This
> >>>> attribute is not defined for SDP session level.
> >>>>
> >>>> The first question we had is if we should base CLUE simulcast
> >>>> support on this individual draft based on CLUE timeline or try to
> >>>> define a different multicast solution at the CLUE protocol level.
> >>>>
> >>>> If we base our solution on the avtcore-rtp-simulcast-04 document
> >>>> the advertisement will include an individual encoding that will
> >>>> relate to optional multiple simulcast encoding and the configure
> >>>> will select such encoding. The actual number of different stream
> >>>> that will compose the simulcast stream will be negotiated using
> >>>> Offer/Answer as described in the above document. The risk here is
> >>>> that this is an individual draft and we may need to review the CLUE
> >>>> support if it changes. The advantage is that we will have simulcast
> >>>> the same for CLUE and non-CLUE systems.
> >>>>

---

From: Paul Kyzivat <[email-masked]>
Date: Sun, 26 Oct 2014 17:33:47 -0400

> other with only HD resolutions

This works when the advertisement assigns the capture to an encoding 
group containing multiple encodings. When the consumer configures the 
same encoding twice, using different encodings from the encoding group, 
then he knows - by definition - they are simulcasts.

	Thanks,
	Paul

---

From: "Roni Even" <[email-masked]>
Date: Mon, 27 Oct 2014 00:00:06 +0200

Paul,
How does the consumer know that he can configure an MC with two encodings,
and which of the encodings are allowed to co-exist?
Roni

> From: Paul Kyzivat [mailto:[email-masked]]
> Sent: 26 October, 2014 11:34 PM
> To: Roni Even; [email-masked]
> >> video, some do small videos. It puts all of those in an encoding
> >> *simulcast*! The advertiser didn't have to do anything special -
> >> didn't have to think about simulcast at all.
> > advertise MC1 with ENC1 and ENC2, in this case the encoding groups
> > will represent different encoding parameters but will need to be
> > defined accordingly, for example one encoding group that has CIF
> > resolutions and the other with only HD resolutions
> 
> This works when the advertisement assigns the capture to an encoding group
> containing multiple encodings. When the consumer configures the same
> encoding twice, using different encodings from the encoding group, then he
> knows - by definition - they are simulcasts.
> 
> 	Thanks,
> 	Paul
> 
> >>> I believe CLUE already supports simulcast today, basically as
> >>> Christian
> >> describes.
> >>>
> >>> More below.
> >>>
> >>>>> 4> seems like the most straight forward approach as it keeps
> >>>>> 4> things
> >>> [Duckworth, Mark] Yes, a media provider can support multiple
> >>> encodings for the same MC, and a consumer can choose to configure
> >>> multiple encodings for the same MC.  You don't even need multiple
> >>> encoding groups to do this, you can do it with just one group that
> >>> contains
> > multiple
> >> encodings.
> >>>
> >>>> You also lose here the option of the total BW that is an encoding
> >>>> group
> >> attribute.
> >>>
> >>>> All these means that we need some changes to CLUE and we can look
> >>>> at what will make the best solution using MCs or MMCs.
> >>>
> >>>>>> Here is my take on the options, my description may not be
> >>>>>> accurate and the purpose here is to continue the discussion
> >>>>>> allowing us to have a better view of the options.
> >>>>>>
> >>>>>> Looking at the state of simulcast support in other WGs, there is
> >>>>>> an individual draft
> >>>>>> unified plan where all encoded streams composing alternatives of
> >>>>>> a simulcast stream are in the same m-line. The alternatives are
> >>>>>> that this is an individual draft and we may need to review the
> >>>>>> CLUE support if it changes. The advantage is that we will have
> >>>>>> simulcast the same for CLUE and non-CLUE systems.
> >>>>>>
> >>>>>> The other option for simulcast support in the CLUE protocol can
> >>>>>> be based on having separate Media Capture for each of the
> >>>>>> different simulcast encoding and use MCC to compose the simulcast
> alternatives.

---

From: Paul Kyzivat <[email-masked]>
Date: Sun, 26 Oct 2014 19:36:50 -0400

> and which of the encodings are allowed to co-exist?

Look at the data model definition of a capture. It has a field 
<maxCaptureEncodings>. That says how many simultaneous encodings of the 
capture are allowed.

So, if the encoding group for the capture includes more than one 
encoding, and maxCaptureEncodings is greater than one, then the consumer 
can configure simulcast.

	Thanks,
	Paul

---

From: "Roni Even" <[email-masked]>
Date: Mon, 27 Oct 2014 06:54:11 +0200

Hi Paul,
That was the first part of the question, now assume that you have an
encoding group of 5 individual encoding and maxCaptuerEncodings is 3, are
all combinations OK?

The AVTcore draft has a specific definition of the simulcast structure that
allows you to define the allowed combinations

Do we say that each capture belongs to a separate encoding group in which
case it is not an encoding group but a multicast group

Roni

> Sent: 27 October, 2014 1:37 AM
> > How does the consumer know that he can configure an MC with two
> > encodings, and which of the encodings are allowed to co-exist?
> 
> So, if the encoding group for the capture includes more than one encoding,
and
> maxCaptureEncodings is greater than one, then the consumer can configure
> simulcast.
> 
> 	Thanks,
> 	Paul
> 
> 
> > Roni
> >
> >>> Marks, suggested different encoding group for each encoding so he
> >>> can advertise MC1 with ENC1 and ENC2, in this case the encoding
> >>> groups will represent different encoding parameters but will need to
> >>> be defined accordingly, for example one encoding group that has CIF
> >> same encoding twice, using different encodings from the encoding
> >> group, then he knows - by definition - they are simulcasts.
> >>
> >> 	Thanks,
> >> 	Paul
> >>
> >>>>> (trying to use both the "correct indentation with > > symbols",
> >>>>> and also marking comments with my name :-)
> >>>>>
> >>>>> Mark
> >>>>>
> >>>>>>> <http://tools.ietf.org/html/draft-westerlund-avtcore-rtp-simulca
> >>>>>>> st
> >>>>>>> -0
> >>>>> Isn't this separate from CLUE, and can be used with CLUE without
> >>>>> any further specification in CLUE documents?
> >>>>>
> >>>>>>> The purpose of simulcast is to provide different encoding
> >>>>>>> options for a particular media stream. A media provider can
> >>>>>>> Advertise a MC
> >>> with
> >>>>>>> A media consumer if it wants to receive multiple encodings for
> >>>>>>> the same MC can simply choose multiple capture encodings e.g.
> >>>>>>> MC1/Encoding1
> >>>>>>> MC1/Encoding2 etc.
> >>>>>
> >>>>> [Duckworth, Mark] By the MC attribute "max capture encodings".
> >>>>> Any encodings in the MC's encoding groups can be configured for a
> >>>>> [Duckworth, Mark] You can still use that by advertising an MC with
> >>>>> a single encoding group, with multiple encodings in the group.
> >>>>>
> >>>>>> solution even though it is not a WG draft, having it as a
> >>>>>> normative reference will delay the publication of the CLUE draft
> >>>>>>
> >>>>>>>
> >>>>>>> Regards, Christian
> >>>>>>>
> >>>>>>>> Looking at the state of simulcast support in other WGs, there
> >>>>>>>> is an individual draft
> >>>>>>>> http://tools.ietf.org/html/draft-westerlund-avtcore-rtp-simulca
> >>>>>>>> st
> >>>>>>>> -0
> >>>>>>>> unified plan where all encoded streams composing alternatives
> >>>>>>>> of a simulcast stream are in the same m-line. The alternatives
> >>>>>>>> are described using an SDP media level attribute "a=simulcast".
> >>>>>>>> This attribute is not defined for SDP session level.
> >>>>>>>>
> >>>>>>>> support on this individual draft based on CLUE timeline or try
> >>>>>>>> to define a different multicast solution at the CLUE protocol
level.
> >>>>>>>>
> >>>>>>>> If we base our solution on the avtcore-rtp-simulcast-04
> >>>>>>>> document the advertisement will include an individual encoding
> >>>>>>>> that will relate to optional multiple simulcast encoding and
> >>>>>>>> the configure will select such encoding. The actual number of
> >>>>>>>> different stream that will compose the simulcast stream will be
> >>>>>>>> negotiated using Offer/Answer as described in the above
> >>>>>>>> document. The risk here is that this is an individual draft and
> >>>>>>>> we may need to review the CLUE support if it changes. The
> >>>>>>>> advantage is that we will have simulcast the same for CLUE and
non-
> CLUE systems.
> >>>>>>>> description and to provide the actual allowed individual
> >>>>>>>> encodes combinations for the Configuration by the consumer.
> >>>>>>>>
> >>>>>>>> Thanks
> >>>>>>>>
> >>>>>>>> Roni
> >>>>>

---

From: "Roni Even" <[email-masked]>
Date: Mon, 27 Oct 2014 12:20:31 +0200

Hi Paul,

If we want to be able to differentiate between simulcast and switched
capture, for example if we have a three camera system that can send three
MCs each with two encodings (simulcast) and a switched capture (loudest
speaker or two loudest speakers out of three) using the same simulcast
option for the each MC currently sent. In this case we can have an MC with
two encodings and an MCC with the three MCs. This will require us to clarify
that having multiple individual encodings in MC is simulcast and not
switched and in MCC it is switched and not simulcast.

Now what may complicate the advertisement is if the individual MCs can
simulcast two encodings while the MCC (switched) is only with one encoding,
this may require different MCs for MCC or individual MCs case

As for the data model draft, the example
https://tools.ietf.org/html/draft-ietf-clue-data-model-schema-07#section-26
may be confusing.
It has for VC0, VC1 and VC2 <maxCaptureEncodings>2</maxCaptureEncodings>
while the EG0 has only three encID so if it was meant to support simulcast
it will require six encID for Scene View SE1. 

Roni

and
level.
non-

---

From: Christian Groves <[email-masked]>
Date: Tue, 28 Oct 2014 11:55:44 +1100

Hello Roni,

As I mentioned in my original reply I think the RTP based simulcast 
method is probably the cleanest approach. There would be nothing 
stopping a CLUE endpoint implementing it when it becomes available (like 
many other RTP features). Using CLUE for simulcast I think is possible 
for most scenarios. The more complicated the scenarios the more 
MCs/encodings an endpoint will need to describe the situation.

Please see my response below.

Regards, Christian

> switched and in MCC it is switched and not simulcast.
[CNG] I don't understand exactly what the issue is here? Could you use 
some pseudo syntax to explain what you're wanting to advertise? You say 
in the first sentence the switched capture uses simulcast but then in 
the last sentence you say the MCC is switched and not simulcast. I would 
think that an MCC can still be switched and simulcast, i.e. the endpoint 
simulcasts whatever source is switched into the media stream.
If you want a pure switched MCC just set maxCaptureEncodings to 1 
against the MCC.
If you want a simultcast MCC just set maxCaptureEncodings to >=2 against 
the MCC.
> this may require different MCs for MCC or individual MCs case.
[CNG] Probably.
> it will require six encID for Scene View SE1.
[CNG] Yes I think you're right. Its worth clarifying this. We could add 
some text in the introduction of the example saying simulcast is used.