Subject: [clue] Question about encoding values

From: "Roni Even" <[email-masked]>
Date: Wed, 25 Jul 2012 11:12:29 +0200

Hi,

I am still trying to understand how the encoding groups and Id work.

Suppose the advertisement include four media captures.

VC1 - left camera

VC2 - center camera

VC3 - right camera

VC4 - switched 

The capture scene entries may be

(vc1,vc2,vc3,vc4)

(vc4)

VC4 appears in the both entries but it may have different encoding
constrains in both cases.

The encoding constrains have two values that reflect different resources.
One is for network bandwidth and the second is for the codec performance.

In the first capture scene entry VC4 does not require any additional codec
resource since it will use one from VC1,2 or 3. As for the network bandwidth
it may use if sent twice or may not need any resource if just mapped.

In the second capture scene entry VC4 will need resources.

How do you specify it with the encoding groups.

Thanks

Roni

---

From: Andy Pepperell <[email-masked]>
Date: Wed, 25 Jul 2012 10:30:41 +0100

Hi Roni,

This is definitely an interesting case, with several different aspects to
it:

1) I don't believe you'd want to set up the capture scene entries in quite
that way; specifically, it's not really "valid" to put VC4 in the same
capture scene entry as VC1, VC2 and VC3, as VC4 is an alternative scene
representation to the VC1+VC2+VC3 entry.

2) You say that VC4 doesn't need any additional resource as it's just one
of VC1, VC2 or VC3. I'd add a caveat to this saying that CLUE doesn't
specify this one way or the other - it may be that a switched VC4 such as
you cite *is* a separate encoding, and we need to ensure the framework
handles both cases well (for instance, if there are 6 seats in total, VC4
might be an encoding of any 2 seats, not just the 1+2, 3+4, 5+6 implied by
the VC1, VC2 and VC3 capture scene entry).

3) There are encode and decode side issues here - on the decode side you
would need to set up 4 decoders if you want to receive all of VC1, VC2, VC3
and VC4 as you couldn't guarantee that VC4 was just one of the others. If
this was so, the consumer would then need to ensure that the summed encode
characteristics of all 4 capture encodings didn't exceed the consumer's
decode limits.

4) Perhaps a more optimal scenario for a consumer that wants to receive
VC1, VC2 and VC3 and have a single capture switched representation of the
same scene, would be to receive (and decode) VC1, VC2 and VC3 (each using
up to 1/3 of the consumer's decoding capability) and performing the segment
switching itself. This would seem to be the only way to guarantee to
achieve what you seem to be implying is always true, namely that VC4 is
just a copy somehow of one of VC1, VC2 and VC3.

My answer to what the consumer would ask for in this case would be that, if
it's decoding capability was high enough, it would be perfectly valid to
ask for each of VC1, VC2, VC3 and VC4 at up to either the encoding group's
maximum or 1/3 (specifically not 1/4) of that - in this scenario the
provider would be able to dynamically split the encode capability either
across 4 separate encodes or 3, depending on whether in fact VC4 was a
simple copy of one of the other captures or a new capture.

>How do you specify it with the encoding groups?

I think this isn't the most complex / subtle part of the question, but I
think it would be up to the provider to ensure that the encoding group
allowed up to 4 encodings (if it was happy to provide VC4 at the same time
as VC1, VC2, and VC3) or only 3 (if it were not). In the former case, each
encoding within the encoding group would need to allow up to at least 1/3
of the group's total capability in order for the case of VC4 being a copy
of one of the other 3 to work optimally. I would re-iterate though that I
do not believe there's any implicit "requirement" or expectation in the
example we're talking about here for VC4 to be just a copy of VC1, VC2 or
VC3, and the framework currently allows it to be a completely separate
capture (/ encoding) - the capture's switched attribute means that it
"represents the (dynamic) most appropriate subset of a 'whole'" according
to the current framework draft.

Hope this helps!

Andy

On Wed, Jul 25, 2012 at 10:12 AM, Roni Even <[email-masked]> wrote:

> Hi,****
>
> I am still trying to understand how the encoding groups and Id work.****
>
> ** **
>
> Suppose the advertisement include four media captures.****
>
> ** **
>
> VC1 – left camera****
>
> VC2 – center camera****
>
> VC3 – right camera****
>
> VC4 – switched ****
>
> ** **
>
> The capture scene entries may be****
>
> ** **
>
> (vc1,vc2,vc3,vc4)****
>
> (vc4)****
>
> ** **
>
> constrains in both cases.****
>
> ** **
>
> One is for network bandwidth and the second is for the codec performance.*
> ***
>
> ** **
>
> resource since it will use one from VC1,2 or 3. As for the network
> bandwidth it may use if sent twice or may not need any resource if just
> mapped.****
>
> In the second capture scene entry VC4 will need resources.****
>
> ** **
>
> How do you specify it with the encoding groups.****
>
> ** **
>
> Thanks****
>
> Roni****
>
> _______________________________________________
> clue mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/clue
>
>

---

From: "Roni Even" <[email-masked]>
Date: Wed, 25 Jul 2012 13:41:36 +0200

Hi Andy,

Thanks, see inline

Roni

From: Andy Pepperell [mailto:[email-masked]] 
Sent: 25 July, 2012 11:31 AM
To: Roni Even
Cc: [email-masked]
Subject: Re: [clue] Question about encoding values

Hi Roni,

RE: As far as I read the framework there is nothing that says that a media
capture can only appear in on capture set entry. The only limitation is that
the MC must be mutually inclusive.

For example: you can have the following capture scene

(VC1,VC2)

(VC1,VC3)

(VC2,VC3)

(VC2,VC4)

(VC4)

2) You say that VC4 doesn't need any additional resource as it's just one of
VC1, VC2 or VC3. I'd add a caveat to this saying that CLUE doesn't specify
this one way or the other - it may be that a switched VC4 such as you cite
*is* a separate encoding, and we need to ensure the framework handles both
cases well (for instance, if there are 6 seats in total, VC4 might be an
encoding of any 2 seats, not just the 1+2, 3+4, 5+6 implied by the VC1, VC2
and VC3 capture scene entry).

RE: This is true and I think this is what I said. The consumer also have no
way to know based on the advertisement if this is a separate or same stream
as one of the others in the capture scene entry.

RE: True, and this is a comment I had to media-9 in
draft-lennox-clue-rtp-usage-04

4) Perhaps a more optimal scenario for a consumer that wants to receive VC1,
VC2 and VC3 and have a single capture switched representation of the same
scene, would be to receive (and decode) VC1, VC2 and VC3 (each using up to
1/3 of the consumer's decoding capability) and performing the segment
switching itself. This would seem to be the only way to guarantee to achieve
what you seem to be implying is always true, namely that VC4 is just a copy
somehow of one of VC1, VC2 and VC3.

RE: it is true but we are talking about the protocol and what is allowed.
Also you are looking from the consumer side and not from the advertisement
side

My answer to what the consumer would ask for in this case would be that, if
it's decoding capability was high enough, it would be perfectly valid to ask
for each of VC1, VC2, VC3 and VC4 at up to either the encoding group's
maximum or 1/3 (specifically not 1/4) of that - in this scenario the
provider would be able to dynamically split the encode capability either
across 4 separate encodes or 3, depending on whether in fact VC4 was a
simple copy of one of the other captures or a new capture.

RE: I think that the consumer cannot know according to the current switched
attribute if VC4 is one of the other VCs or something else. Only the
provider will know and this should be reflected in the encoding information
advertisement.

I think this isn't the most complex / subtle part of the question, but I
think it would be up to the provider to ensure that the encoding group
allowed up to 4 encodings (if it was happy to provide VC4 at the same time
as VC1, VC2, and VC3) or only 3 (if it were not). In the former case, each
encoding within the encoding group would need to allow up to at least 1/3 of
the group's total capability in order for the case of VC4 being a copy of
one of the other 3 to work optimally. I would re-iterate though that I do
not believe there's any implicit "requirement" or expectation in the example
we're talking about here for VC4 to be just a copy of VC1, VC2 or VC3, and
the framework currently allows it to be a completely separate capture (/
encoding) - the capture's switched attribute means that it "represents the
(dynamic) most appropriate subset of a 'whole'" according to the current
framework draft.

RE: so what understand is that the advertisement in the encoding group
should allow for VC4 to use an entry that will allow for VC4 sent alone and
the consumer need to limit he configures VC4 as part of the capture scene
entry with VC1-3

So VC4 for example may be able to do up to 1080P30fps at 4Mbit/sec but when
used with the other VCs the consumer may limit VC4 to SD and 1 Mbit/sec

Hope this helps!

Andy

Hi,

VC1 - left camera

VC2 - center camera

VC3 - right camera

VC4 - switched 

(vc1,vc2,vc3,vc4)

(vc4)

Thanks

Roni

---

From: Andy Pepperell <[email-masked]>
Date: Wed, 25 Jul 2012 15:10:23 +0100

Roni,

Comments from me inline too...

Andy

On Wed, Jul 25, 2012 at 12:41 PM, Roni Even <[email-masked]> wrote:

> Hi Andy,****
>
> Thanks, see inline****
>
> Roni****
>
> ** **
>
> *From:* Andy Pepperell [mailto:[email-masked]]
> *Sent:* 25 July, 2012 11:31 AM
> *To:* Roni Even
> *Cc:* [email-masked]
> *Subject:* Re: [clue] Question about encoding values****
>
> ** **
>
> Hi Roni,****
>
> ** **
>
> it:****
> representation to the VC1+VC2+VC3 entry.****
>
> capture can only appear in on capture set entry. The only limitation is
> that the MC must be mutually inclusive.
>

Yes, agreed

> ****
>
> For example: you can have the following capture scene****
>
> (VC1,VC2)****
>
> (VC1,VC3)****
>
> (VC2,VC3)****
>
> (VC2,VC4)****
>
> (VC4)
>

That's definitely possible, but it's also true that each capture scene
entry should be a representation of the whole scene, i.e. the semantics are
that whichever entry you choose you get a representation of the whole
scene. That wouldn't be true I think for this case - what your example
capture scene is saying is that VC1+VC2 gives not only a representation of
the whole scene, but also that VC1+VC does (and all the other entries).

> ****
> the VC1, VC2 and VC3 capture scene entry).****
>
> ** **
>
> RE: This is true and I think this is what I said.
>

>In the first capture scene entry VC4 does not require any additional codec
resource**

>In the second capture scene entry VC4 will need resources.

I'm not sure that there's any such distinction conveyed by the framework -
i.e. I don't think it's the case that whether VC4 requires extra resources
depends on whether it appears with VC1..3 or not, nor is it guaranteed that
this would be true even just looking at the provider itself (i.e. with no
consideration to how it advertises its captures to the far end via the CLUE
framework) as the provider side of some systems might work one way and
others a different way.

> The consumer also have no way to know based on the advertisement if this
> is a separate or same stream as one of the others in the capture scene
> entry.
>

I suppose my response to this would be that if that was a necessary
characteristic then the only way to guarantee it would be for the consumer
to perform the switch, i.e. decide to receive just VC1..3 and determine
which of those captures to show. If there was only a single decoder /
screen, then it could receive just VC4 and so get the switching that way.

> ****
> decode limits.****

I think that the media-9 requirement in "draft-lennox-clue-rtp-usage-04"
applies here, yes, and would seem to work just fine, with the proviso that
while the consumer would be able to detect that capture encoding of VC4 was
a copy of one of the other video captures, it wouldn't necessarily be able
to rely on that (always) being so.

> ****
> just a copy somehow of one of VC1, VC2 and VC3.****
>
> ** **
>

I guess that depends on what you're trying to do. Is it the case that
you're trying to find a way for the framework / protocol to explicitly
signal that the switched capture *is* tied to being one of the constituent
captures from a different capture scene entry?

> ****
>
> ** **
>
> My answer to what the consumer would ask for in this case would be that,
> if it's decoding capability was high enough, it would be perfectly valid to
> simple copy of one of the other captures or a new capture.****
>
> RE: I think that the consumer cannot know according to the current
> switched attribute if VC4 is one of the other VCs or something else. Only
> the provider will know and this should be reflected in the encoding
> information advertisement.
>

It's true that only the provider will know - it's not 100% clear to me
though that there needs to be any mechanism / signaling over and above
what's already defined to be able to handle this case. Which cases are you
thinking of that would go wrong in the current set up? I am starting to
think that this may lead us into an SSRC discussion too (in some cases
VC4's SSRC will change according to which of VC1..3 it represents, in
others there'll be a single persistent VC4 SSRC).

> ****
>
> ** **
>
> >How do you specify it with the encoding groups?****
>
> ** **
>
> to the current framework draft.****
>
> ** **
>
> entry with VC1-3****
> So VC4 for example may be able to do up to 1080P30fps at 4Mbit/sec but
> when used with the other VCs the consumer may limit VC4 to SD and 1 Mbit/sec
>

I'm not sure if you're asking a question here, pointing out something that
doesn't work, or saying you're happy with the current state of things (I'm
guessing not the last option though :) ). I think that in the general (/
worst) case the consumer would have to apportion its decoding resources
conservatively to allow for VC4 being a completely separate stream (needing
a separate decode) to any of VC1..3.

> ****
>
> ** **
>
> Hope this helps!****
>
> ** **
>
> Andy****
>
> ** **
>
> ** **
>
> On Wed, Jul 25, 2012 at 10:12 AM, Roni Even <[email-masked]>
> wrote:****
>
> Hi,****
>
>  ****
>  ****
>  ****
>  ****
>  ****
>  ****
>  ****
>  ****
>  ****
> https://www.ietf.org/mailman/listinfo/clue****
>
> ** **
>

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 25 Jul 2012 10:25:36 -0400

On 7/25/12 5:12 AM, Roni Even wrote:
> VC1 – left camera
> VC2 – center camera
> VC3 – right camera
> VC4 – switched
> The encoding constrains have two values that reflect different
> resources. One is for network bandwidth and the second is for the codec
> performance.
>
> In the first capture scene entry VC4 does not require any additional
> codec resource since it will use one from VC1,2 or 3.

I wonder about this. If capture ids are being sent in the media stream, 
then won't a separate codec resource be required for VC4 to do that? Or 
are you assuming that insertion of the capture id is a separate function 
from the codec?

	Thanks,
	Paul

---

From: Andy Pepperell <[email-masked]>
Date: Wed, 25 Jul 2012 15:44:34 +0100

Hi Paul,

>I wonder about this. If capture ids are being sent in the media stream,
then won't a separate codec resource be required for VC4 to do that? Or are
you assuming that insertion of the capture id is a separate function from
the codec?

I think it's true that there will be a separate distinct ID for VC4 as
compared to any of the others (this is obviously essential for cases where
VC4 is not just a simple copy of VC1, VC2 or VC3). The media-9 requirement
that Roni cites in "draft-lennox-clue-rtp-usage-04" covers this case to
some degree, and the intent (from that draft) in this scenario is for the
provider to use more than one header-extension value within the same RTP
stream (most notably, the same SSRC) to indicate this equivalence to the
consumer (the consumer may or may not be set up to be optimised for this
case).

A forthcoming amendment to that RTP usage draft proposes that instead of
the capture ID being used within the RTP encoding, it's a consumer-chosen
ID (for the purposes of this discussion I don't believe it's significant,
but this is necessary for simulcast cases, and at the last interim meeting
there was broad agreement over the usefulness of the ID being chosen by the
consumer-side rather than the provider).

Andy

On Wed, Jul 25, 2012 at 3:25 PM, Paul Kyzivat <[email-masked]> wrote:

> then won't a separate codec resource be required for VC4 to do that? Or are
> you assuming that insertion of the capture id is a separate function from
> the codec?
>
>         Thanks,
>         Paul
>
> ______________________________**_________________
> clue mailing list
> [email-masked]
> https://www.ietf.org/mailman/**listinfo/clue<https://www.ietf.org/mailman/listinfo/clue>
>

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 25 Jul 2012 10:55:29 -0400

On 7/25/12 10:44 AM, Andy Pepperell wrote:
> Hi Paul,
>
>  >I wonder about this. If capture ids are being sent in the media
> stream, then won't a separate codec resource be required for VC4 to do
> that? Or are you assuming that insertion of the capture id is a separate
> function from the codec?
>
> compared to any of the others (this is obviously essential for cases
> where VC4 is not just a simple copy of VC1, VC2 or VC3). The media-9
> requirement that Roni cites in "draft-lennox-clue-rtp-usage-04" covers
> this case to some degree, and the intent (from that draft) in this
> scenario is for the provider to use more than one header-extension value
> within the same RTP stream (most notably, the same SSRC) to indicate
> this equivalence to the consumer (the consumer may or may not be set up
> to be optimised for this case).
>
> the capture ID being used within the RTP encoding, it's a
> consumer-chosen ID (for the purposes of this discussion I don't believe
> it's significant, but this is necessary for simulcast cases, and at the
> last interim meeting there was broad agreement over the usefulness of
> the ID being chosen by the consumer-side rather than the provider).

I understand all that. But you didn't answer my question:

Does this mean that a switched capture always requires a separate 
encoding resource?

I guess you were arguing that the consumer must assume it does.

	Thanks,
	Paul

> Andy
>
>
>
> On Wed, Jul 25, 2012 at 3:25 PM, Paul Kyzivat <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     I wonder about this. If capture ids are being sent in the media
>     stream, then won't a separate codec resource be required for VC4 to
>     do that? Or are you assuming that insertion of the capture id is a
>     separate function from the codec?
>
>              Thanks,
>              Paul
>
>     _________________________________________________
>     clue mailing list
>     [email-masked] <mailto:[email-masked]>
>     https://www.ietf.org/mailman/__listinfo/clue
>     <https://www.ietf.org/mailman/listinfo/clue>
>
>

---

From: Andrew Pepperell <[email-masked]>
Date: Wed, 25 Jul 2012 18:50:00 +0100

Hi Paul,

Apologies if it seemed like I was evading your question (!). Partly I was keen not to give the impression what I was saying was any more than just my opinion on this issue...

In the general case I believe VC4 in this example has to [permitted to be] a separately encoded stream needing a separate decode. This would be essential for cases where the switched sub-scene view *wasn't* just one of VC1..3, and also for cases where the consumer requires different encoding characteristics, e.g. a lower resolution or bit rate.

The RTP usage draft from Jonathan and others describes how the same actual encoding can be sent by a provider to satisfy multiple consumer choices. I maintain, though clearly it's a matter of opinion, that the best way for a consumer to guarantee that "VC4" be a shared encode of one of VC1..VC3 is for it to receive only those captures and perform the segment switching itself (we've talked in various meetings about linking VAD / energy detection to provider-side co-ordinates and this would seem to be applicable here, as it also addresses cases where the provider would not be able to supply a separate, additional, VC4 equivalent).

You probably think this still doesn't answer your question - I think for some cases it's yes and for some it's no, depending on how the provider is providing the switched capture and on how the consumer plans to use it. It's definitely the case that ghere's no explicit signalling for a provider to say "capture N" is always one of this set of other captures - by way of further generalization, it might also be instructive to consider a potential additional capture scene entry consisting of VC5..VC8, a 4 capture representation of the scene. If this too were present, it would not be possible to determine without extra explicit indication from the provider whether the switched VC4 was one of the 1..3 set or the 5..8 set.

Regards,

Andy

On 25 Jul 2012, at 15:55, Paul Kyzivat <[email-masked]> wrote:

> Does this mean that a switched capture always requires a separate encoding resource?
>

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 26 Jul 2012 10:45:42 -0400

> You probably think this still doesn't answer your question - I think for some cases it's yes and for some it's no, depending on how the provider is providing the switched capture and on how the con…

ISTM there are *two* questions hidden here, though they are closely related.

One is whether VC4 requires a separate *encoding* resource.
The 2nd is whether VC4 requires a separate *decoding* resource.

The question of the encoding resource comes down to a decision by the 
receiver about which combinations of captures can be selected. If the 
receiver thinks that a separate encoder is required for VC4, then it may 
conclude that it won't be able to select it with some other captures 
because the advertiser hasn't indicated sufficient encoding resources.

 From the provider's perspective, it may be that it can perform an 
internal optimization and so require less resources than the receiver 
thinks are required. That is nice for the provider, but it also means 
that some useful configurations aren't usable because they can't be 
discovered by the recipient.

Similarly, the recipient can only select those configurations that it 
has decoders to handle. And once it has assigned decoders, it probably 
has less opportunity (than the provider) to exploit optimizations that 
would let it use fewer decoders.

Having said all that, I don't know if it implies any changes to the 
mechanisms that have proposed.

	Thanks,
	Paul

> Regards,
>
> Andy
>
>
> On 25 Jul 2012, at 15:55, Paul Kyzivat<[email-masked]>  wrote:
>
>>> On Wed, Jul 25, 2012 at 3:25 PM, Paul Kyzivat<[email-masked]
>>>     [email-masked]<mailto:[email-masked]>

---

From: "Roni Even" <[email-masked]>
Date: Fri, 27 Jul 2012 10:10:29 +0200

Hi,
There are another questions. 
If VC4 does not require extra decoding resources, it will still require it when VC4 is advertised as a separate media capture entry. 
Can I provide a different limit on VC4 depending on which media capture entry uses it. This is a general question when a media capture is advertised in more than one media capture entry but the advertisement would like different limitation on it. I assume the only way is to not have the same media capture in more than one entry and the text should mention this issue.
This will mean that in my example you cannot have both as VC4 and in the first entry it must be possible to say that there are no limitation on the MC in the advertisement to make it clear that no extra resources are needed for it when switching between VC1-3.
Roni

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Paul Kyzivat
Sent: 26 July, 2012 4:46 PM
To: Andrew Pepperell
Cc: [email-masked]
Subject: Re: [clue] Question about encoding values

The question of the encoding resource comes down to a decision by the receiver about which combinations of captures can be selected. If the receiver thinks that a separate encoder is required for VC4, then it may conclude that it won't be able to select it with some other captures because the advertiser hasn't indicated sufficient encoding resources.

 From the provider's perspective, it may be that it can perform an internal optimization and so require less resources than the receiver thinks are required. That is nice for the provider, but it also means that some useful configurations aren't usable because they can't be discovered by the recipient.

Similarly, the recipient can only select those configurations that it has decoders to handle. And once it has assigned decoders, it probably has less opportunity (than the provider) to exploit optimizations that would let it use fewer decoders.

Having said all that, I don't know if it implies any changes to the mechanisms that have proposed.

	Thanks,
	Paul

>>> I think it's true that there will be a separate distinct ID for VC4 
>>> as compared to any of the others (this is obviously essential for 
>>> cases where VC4 is not just a simple copy of VC1, VC2 or VC3). The 
>>> media-9 requirement that Roni cites in 
>>> "draft-lennox-clue-rtp-usage-04" covers this case to some degree, 
>>> and the intent (from that draft) in this scenario is for the 
>>> provider to use more than one header-extension value within the same 
>>> RTP stream (most notably, the same SSRC) to indicate this 
>>> equivalence to the consumer (the consumer may or may not be set up to be optimised for this case).
>>>
>>> A forthcoming amendment to that RTP usage draft proposes that 
>>> instead of the capture ID being used within the RTP encoding, it's a 
>>> consumer-chosen ID (for the purposes of this discussion I don't 
>>> believe it's significant, but this is necessary for simulcast cases, 
>>> and at the last interim meeting there was broad agreement over the 
>>> usefulness of the ID being chosen by the consumer-side rather than the provider).
>>

---

From: Andy Pepperell <[email-masked]>
Date: Fri, 27 Jul 2012 08:40:09 +0100

Hi Roni,

>If VC4 does not require extra decoding resources, it will still require it
when VC4 is advertised as a separate media capture entry.

I'm not sure that which capture scene entry VC4 is part of is a determining
factor here - as currently defined, VC4 shouldn't be in the same capture
scene entry as VC1..3 as it's an alternative to their capture scene entry
rather than being part of it.

>Can I provide a different limit on VC4 depending on which media capture
entry uses it

I'm not sure how best to answer this, as, again, to my mind it wouldn't
depend on which capture scene entry VC4 was to be found in (as it should
always be separate to VC1..VC3) but different limits are supported through
the encoding groups - as it currently stands, the way to do what you're
describing through the framework would be to have VC4 in a different
encoding group.

>This is a general question when a media capture is advertised in more than
one media capture entry but the advertisement would like different
limitation on it.
>I assume the only way is to not have the same media capture in more than
one entry and the text should mention this issue.

I'm not sure that having a media capture in more than one capture scene
entry makes much sense within the framework as it currently stands; to
re-iterate: capture scene entries are alternative representation of the
same capture scene and so having a media capture entry in more than one
capture scene entry would, I think, make it more difficult for a consumer
to be able to make a sensible choice.

It seems to me that what this discussion is about is introducing a new
semantic whereby if you put one or more switched media captures in the same
capture scene entry as one or more non-switched captures then there's an
implication that the switched captures are just copies of the others (and
so don't need separate decoding, and affects how the consumer choices need
to fit in with the encoding [group] limits as declared by the provider).
This is not something covered by the framework at the moment, which
intentionally focusses on the more general cases where a switched version
either isn't a simple copy of one of the non-switched captures (e.g. it's a
separate PTZ camera) or a consumer wants to receive it with different
encoding characteristics (bit rate, resolution etc.). I would go back
(perhaps tediously!) to an earlier comment where if a consumer really wants
to do segment switching and not separately decode the switched capture,
guaranteeing that the switched version is a copy of one of the other
streams, the optimal way to do this would be for it to perform the segment
switching itself. This would also have the advantage of working with
providers which either didn't offer the segment switched capture at all or
did so in a way that would definitely require a separate decode.

Regards,

Andy

On Fri, Jul 27, 2012 at 9:10 AM, Roni Even <[email-masked]> wrote:

> If VC4 does not require extra decoding resources, it will still require it
> when VC4 is advertised as a separate media capture entry.
> Can I provide a different limit on VC4 depending on which media capture
> entry uses it. This is a general question when a media capture is
> advertised in more than one media capture entry but the advertisement would
> like different limitation on it. I assume the only way is to not have the
> same media capture in more than one entry and the text should mention this
> issue.
> This will mean that in my example you cannot have both as VC4 and in the
> first entry it must be possible to say that there are no limitation on the
> MC in the advertisement to make it clear that no extra resources are needed
> for it when switching between VC1-3.
> Roni
>
> > Apologies if it seemed like I was evading your question (!). Partly I
> was keen not to give the impression what I was saying was any more than
> just my opinion on this issue...
> >
> > In the general case I believe VC4 in this example has to [permitted to
> be] a separately encoded stream needing a separate decode. This would be
> essential for cases where the switched sub-scene view *wasn't* just one of
> VC1..3, and also for cases where the consumer requires different encoding
> characteristics, e.g. a lower resolution or bit rate.
> >
> > The RTP usage draft from Jonathan and others describes how the same
> actual encoding can be sent by a provider to satisfy multiple consumer
> choices. I maintain, though clearly it's a matter of opinion, that the best
> way for a consumer to guarantee that "VC4" be a shared encode of one of
> VC1..VC3 is for it to receive only those captures and perform the segment
> switching itself (we've talked in various meetings about linking VAD /
> energy detection to provider-side co-ordinates and this would seem to be
> applicable here, as it also addresses cases where the provider would not be
> able to supply a separate, additional, VC4 equivalent).
> >
> > You probably think this still doesn't answer your question - I think for
> some cases it's yes and for some it's no, depending on how the provider is
> providing the switched capture and on how the consumer plans to use it.
> It's definitely the case that ghere's no explicit signalling for a provider
> to say "capture N" is always one of this set of other captures - by way of
> further generalization, it might also be instructive to consider a
> potential additional capture scene entry consisting of VC5..VC8, a 4
> capture representation of the scene. If this too were present, it would not
> be possible to determine without extra explicit indication from the
> provider whether the switched VC4 was one of the 1..3 set or the 5..8 set.
>
> thinks are required. That is nice for the provider, but it also means that
> some useful configurations aren't usable because they can't be discovered
> by the recipient.
>
> Similarly, the recipient can only select those configurations that it has
> decoders to handle. And once it has assigned decoders, it probably has less
> opportunity (than the provider) to exploit optimizations that would let it
> use fewer decoders.
>

---

From: "Roni Even" <[email-masked]>
Date: Fri, 27 Jul 2012 11:20:27 +0200

Andy,

The problem is that the framework does not prohibit having a MC in two
capture scene entries.

Example of use case that may make more sense is when VC2 is center camera

(VC1, VC2, VC3) - to a three monitor

(VC4 (switched between VC1 and VC3), VC2) - to two monitor.

Again in this case the encoding capabilities between the first and second
entry for VC2 may be different and we cannot show it in the framework using
encoding groups  unless you have a different MC for the center camera for
each capture scene entry. The encoding group as is will provide the same
maximum for both.

I think that we need to clarify this point in the framework.

Roni

From: Andy Pepperell [mailto:[email-masked]] 
Sent: 27 July, 2012 9:40 AM
To: Roni Even
Cc: Paul Kyzivat; [email-masked]
Subject: Re: [clue] Question about encoding values

Hi Roni,

entry uses it 

I'm not sure that having a media capture in more than one capture scene
entry makes much sense within the framework as it currently stands; to
re-iterate: capture scene entries are alternative representation of the same
capture scene and so having a media capture entry in more than one capture
scene entry would, I think, make it more difficult for a consumer to be able
to make a sensible choice.

It seems to me that what this discussion is about is introducing a new
semantic whereby if you put one or more switched media captures in the same
capture scene entry as one or more non-switched captures then there's an
implication that the switched captures are just copies of the others (and so
don't need separate decoding, and affects how the consumer choices need to
fit in with the encoding [group] limits as declared by the provider). This
is not something covered by the framework at the moment, which intentionally
focusses on the more general cases where a switched version either isn't a
simple copy of one of the non-switched captures (e.g. it's a separate PTZ
camera) or a consumer wants to receive it with different encoding
characteristics (bit rate, resolution etc.). I would go back (perhaps
tediously!) to an earlier comment where if a consumer really wants to do
segment switching and not separately decode the switched capture,
guaranteeing that the switched version is a copy of one of the other
streams, the optimal way to do this would be for it to perform the segment
switching itself. This would also have the advantage of working with
providers which either didn't offer the segment switched capture at all or
did so in a way that would definitely require a separate decode.

Regards,

Andy

Hi,
There are another questions.
If VC4 does not require extra decoding resources, it will still require it
when VC4 is advertised as a separate media capture entry.
Can I provide a different limit on VC4 depending on which media capture
entry uses it. This is a general question when a media capture is advertised
in more than one media capture entry but the advertisement would like
different limitation on it. I assume the only way is to not have the same
media capture in more than one entry and the text should mention this issue.
This will mean that in my example you cannot have both as VC4 and in the
first entry it must be possible to say that there are no limitation on the
MC in the advertisement to make it clear that no extra resources are needed
for it when switching between VC1-3.
Roni

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Paul
Kyzivat
Sent: 26 July, 2012 4:46 PM
To: Andrew Pepperell
Cc: [email-masked]
Subject: Re: [clue] Question about encoding values

On 7/25/12 1:50 PM, Andrew Pepperell wrote:
> Hi Paul,
>
> Apologies if it seemed like I was evading your question (!). Partly I was
keen not to give the impression what I was saying was any more than just my
opinion on this issue...
>
> In the general case I believe VC4 in this example has to [permitted to be]
a separately encoded stream needing a separate decode. This would be
essential for cases where the switched sub-scene view *wasn't* just one of
VC1..3, and also for cases where the consumer requires different encoding
characteristics, e.g. a lower resolution or bit rate.
>
> The RTP usage draft from Jonathan and others describes how the same actual
encoding can be sent by a provider to satisfy multiple consumer choices. I
maintain, though clearly it's a matter of opinion, that the best way for a
consumer to guarantee that "VC4" be a shared encode of one of VC1..VC3 is
for it to receive only those captures and perform the segment switching
itself (we've talked in various meetings about linking VAD / energy
detection to provider-side co-ordinates and this would seem to be applicable
here, as it also addresses cases where the provider would not be able to
supply a separate, additional, VC4 equivalent).
some cases it's yes and for some it's no, depending on how the provider is
providing the switched capture and on how the consumer plans to use it. It's
definitely the case that ghere's no explicit signalling for a provider to
say "capture N" is always one of this set of other captures - by way of
further generalization, it might also be instructive to consider a potential
additional capture scene entry consisting of VC5..VC8, a 4 capture
representation of the scene. If this too were present, it would not be
possible to determine without extra explicit indication from the provider
whether the switched VC4 was one of the 1..3 set or the 5..8 set.

The question of the encoding resource comes down to a decision by the
receiver about which combinations of captures can be selected. If the
receiver thinks that a separate encoder is required for VC4, then it may
conclude that it won't be able to select it with some other captures because
the advertiser hasn't indicated sufficient encoding resources.

 From the provider's perspective, it may be that it can perform an internal
optimization and so require less resources than the receiver thinks are
required. That is nice for the provider, but it also means that some useful
configurations aren't usable because they can't be discovered by the
recipient.

        Thanks,
        Paul

encoding resource?

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 27 Jul 2012 09:43:48 -0400

It occurs to me that there is another way that the description of 
encoding resources is unable to capture nuances:

In some cases an MCU may be able to send the same encoded media to 
multiple consumers. If so, then that one may be considered "free" to all 
but one of the consumers. Then in principle they could get another 
capture that they otherwise wouldn't be able to get.

The reuse that Roni is talking about is similar, but arises for a single 
provider/consumer relationship.

I guess the challenge is that providing enough information to figure out 
when this applies is probably too difficult for us to attempt. And if 
not we will have to settle for the suboptimal solution that assumes 
every received capture requires a separate encoder.

Alternatively, I guess it would be possible to stop trying to convey 
encoding constraints from provider to consumer, and instead use some 
sort of negotiation to arrive at a workable configuration. But that 
seems unreasonably complex.

	Thanks,
	Paul

On 7/27/12 3:40 AM, Andy Pepperell wrote:
> Hi Roni,
>
>  >If VC4 does not require extra decoding resources, it will still
> require it when VC4 is advertised as a separate media capture entry.
>
> I'm not sure that which capture scene entry VC4 is part of is a
> determining factor here - as currently defined, VC4 shouldn't be in the
> same capture scene entry as VC1..3 as it's an alternative to their
> capture scene entry rather than being part of it.
>
>  >Can I provide a different limit on VC4 depending on which media
> capture entry uses it
>
> always be separate to VC1..VC3) but different limits are supported
> through the encoding groups - as it currently stands, the way to do what
> you're describing through the framework would be to have VC4 in a
> different encoding group.
>
>  >This is a general question when a media capture is advertised in more
> than one media capture entry but the advertisement would like different
> limitation on it.
>  >I assume the only way is to not have the same media capture in more
> than one entry and the text should mention this issue.
>
> capture scene entry would, I think, make it more difficult for a
> consumer to be able to make a sensible choice.
>
> semantic whereby if you put one or more switched media captures in the
> same capture scene entry as one or more non-switched captures then
> there's an implication that the switched captures are just copies of the
> others (and so don't need separate decoding, and affects how the
> consumer choices need to fit in with the encoding [group] limits as
> declared by the provider). This is not something covered by the
> framework at the moment, which intentionally focusses on the more
> general cases where a switched version either isn't a simple copy of one
> of the non-switched captures (e.g. it's a separate PTZ camera) or a
> consumer wants to receive it with different encoding characteristics
> (bit rate, resolution etc.). I would go back (perhaps tediously!) to an
> earlier comment where if a consumer really wants to do segment switching
> and not separately decode the switched capture, guaranteeing that the
> switched version is a copy of one of the other streams, the optimal way
> to do this would be for it to perform the segment switching itself. This
> would also have the advantage of working with providers which either
> didn't offer the segment switched capture at all or did so in a way that
> would definitely require a separate decode.
>
> Regards,
>
> Andy
>
>
> On Fri, Jul 27, 2012 at 9:10 AM, Roni Even <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     Hi,
>     If VC4 does not require extra decoding resources, it will still
>     require it when VC4 is advertised as a separate media capture entry.
>     Can I provide a different limit on VC4 depending on which media
>     capture entry uses it. This is a general question when a media
>     capture is advertised in more than one media capture entry but the
>     advertisement would like different limitation on it. I assume the
>     only way is to not have the same media capture in more than one
>     entry and the text should mention this issue.
>     This will mean that in my example you cannot have both as VC4 and in
>     the first entry it must be possible to say that there are no
>     limitation on the MC in the advertisement to make it clear that no
>     extra resources are needed for it when switching between VC1-3.
>     Roni
>
>     From: [email-masked] <mailto:[email-masked]>
>     [mailto:[email-masked] <mailto:[email-masked]>] On
>     Behalf Of Paul Kyzivat
>     Cc: [email-masked] <mailto:[email-masked]>
>      > Apologies if it seemed like I was evading your question (!).
>     Partly I was keen not to give the impression what I was saying was
>     any more than just my opinion on this issue...
>      >
>      > In the general case I believe VC4 in this example has to
>     [permitted to be] a separately encoded stream needing a separate
>     decode. This would be essential for cases where the switched
>     sub-scene view *wasn't* just one of VC1..3, and also for cases where
>     the consumer requires different encoding characteristics, e.g. a
>     lower resolution or bit rate.
>      >
>      > The RTP usage draft from Jonathan and others describes how the
>     same actual encoding can be sent by a provider to satisfy multiple
>     consumer choices. I maintain, though clearly it's a matter of
>     opinion, that the best way for a consumer to guarantee that "VC4" be
>     a shared encode of one of VC1..VC3 is for it to receive only those
>     captures and perform the segment switching itself (we've talked in
>     various meetings about linking VAD / energy detection to
>     provider-side co-ordinates and this would seem to be applicable
>     here, as it also addresses cases where the provider would not be
>      > You probably think this still doesn't answer your question - I
>     think for some cases it's yes and for some it's no, depending on how
>     the provider is providing the switched capture and on how the
>     consumer plans to use it. It's definitely the case that ghere's no
>     explicit signalling for a provider to say "capture N" is always one
>     of this set of other captures - by way of further generalization, it
>     might also be instructive to consider a potential additional capture
>     scene entry consisting of VC5..VC8, a 4 capture representation of
>     the scene. If this too were present, it would not be possible to
>     determine without extra explicit indication from the provider
>     The question of the encoding resource comes down to a decision by
>     the receiver about which combinations of captures can be selected.
>     If the receiver thinks that a separate encoder is required for VC4,
>     then it may conclude that it won't be able to select it with some
>     other captures because the advertiser hasn't indicated sufficient
>     encoding resources.
>
>     internal optimization and so require less resources than the
>     receiver thinks are required. That is nice for the provider, but it
>     also means that some useful configurations aren't usable because
>     they can't be discovered by the recipient.
>
>     Similarly, the recipient can only select those configurations that
>     it has decoders to handle. And once it has assigned decoders, it
>     probably has less opportunity (than the provider) to exploit
>     optimizations that would let it use fewer decoders.
>
>      > On 25 Jul 2012, at 15:55, Paul Kyzivat<[email-masked]
>      >>> On Wed, Jul 25, 2012 at 3:25 PM, Paul
>     Kyzivat<[email-masked] <mailto:[email-masked]>
>      >>> <mailto:[email-masked] <mailto:[email-masked]>>>
>       wrote:
>      >>>
>      >>>         resources. One is for network bandwidth and the second
>     is for
>      >>>         the codec
>      >>>         performance.
>      >>>
>      >>> [email-masked] <mailto:[email-masked]><mailto:[email-masked]
>     <mailto:[email-masked]>>

---

From: "Roni Even" <[email-masked]>
Date: Sat, 28 Jul 2012 00:43:57 +0200

Paul,
In the MCU case each participant will get an individual advertisement. The
MCU may send the same encoded stream to all using the common denominator
(video switch) or if possible send each one an individual encode
(transcoding), this is done in current MCUs. The request from the receiver
is the maximum value , which is also true for the encoding groups in the
CLUE (at least this is my understanding)
Roni

-----Original Message-----
From: Paul Kyzivat [mailto:[email-masked]] 
Sent: 27 July, 2012 3:44 PM
To: Andy Pepperell
Cc: Roni Even; [email-masked]
Subject: Re: [clue] Question about encoding values

It occurs to me that there is another way that the description of encoding
resources is unable to capture nuances:

In some cases an MCU may be able to send the same encoded media to multiple
consumers. If so, then that one may be considered "free" to all but one of
the consumers. Then in principle they could get another capture that they
otherwise wouldn't be able to get.

I guess the challenge is that providing enough information to figure out
when this applies is probably too difficult for us to attempt. And if not we
will have to settle for the suboptimal solution that assumes every received
capture requires a separate encoder.

Alternatively, I guess it would be possible to stop trying to convey
encoding constraints from provider to consumer, and instead use some sort of
negotiation to arrive at a workable configuration. But that seems
unreasonably complex.

	Thanks,
	Paul

On 7/27/12 3:40 AM, Andy Pepperell wrote:
> determining factor here - as currently defined, VC4 shouldn't be in 
> the same capture scene entry as VC1..3 as it's an alternative to their 
> I'm not sure how best to answer this, as, again, to my mind it 
> wouldn't depend on which capture scene entry VC4 was to be found in 
> (as it should always be separate to VC1..VC3) but different limits are 
> supported through the encoding groups - as it currently stands, the 
> way to do what you're describing through the framework would be to 
> have VC4 in a different encoding group.
>
>  >This is a general question when a media capture is advertised in 
> more than one media capture entry but the advertisement would like 
> different limitation on it.
> I'm not sure that having a media capture in more than one capture 
> scene entry makes much sense within the framework as it currently 
> stands; to
> re-iterate: capture scene entries are alternative representation of 
> the same capture scene and so having a media capture entry in more 
> than one capture scene entry would, I think, make it more difficult 
> for a consumer to be able to make a sensible choice.
>
> there's an implication that the switched captures are just copies of 
> the others (and so don't need separate decoding, and affects how the 
> general cases where a switched version either isn't a simple copy of 
> one of the non-switched captures (e.g. it's a separate PTZ camera) or 
> a consumer wants to receive it with different encoding characteristics 
> (bit rate, resolution etc.). I would go back (perhaps tediously!) to 
> an earlier comment where if a consumer really wants to do segment 
> switching and not separately decode the switched capture, guaranteeing 
> that the switched version is a copy of one of the other streams, the 
> optimal way to do this would be for it to perform the segment 
> providers which either didn't offer the segment switched capture at 
> all or did so in a way that would definitely require a separate decode.
>
> Regards,
>
> Andy
>
>

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Tue, 31 Jul 2012 01:40:26 +0000

It seems to me that while a provider may be able in some cases to make some optimizations when sending the same media to multiple consumers, my assumption is that this is not generally the case, as all it takes is one "fir" message or one "tmmbr" messages and suddenly what the provider is sending one consumer will differ from what it sends others.
I think any optimization or the ability of a provider to perform such optimizations can be left out of CLUE advertisements and configurations.

Cheers,
Charles

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Roni Even
> Sent: Friday, July 27, 2012 3:44 PM
> To: 'Paul Kyzivat'; 'Andy Pepperell'
> Cc: [email-masked]
> multiple