Subject: Re: [clue] negotiation between consumer and provider

From: Robert Hansen <[email-masked]>
Date: Fri, 01 Jun 2012 17:14:33 +0100

Hi Paul, comments inline:
(NB, the UK is off for the next four days to celebrate our queenliness, 
so this will likely be the last email I'll be able to send until Wednesday).

On 31/05/2012 19:05, Paul Kyzivat wrote:
> I just read through the three new drafts:
>
> draft-pepperell-clue-switched-attribute
> draft-hansen-clue-consumer-layout
> draft-romanow-clue-audio-rendering-tag
>
> All three of these propose a mechanism that, AFAIK, is new to CLUE:
> it is that data included by the consumer in its selection message to the
> provider might alter in a fundamental way what content is included in
> the selected captures. More specifically, its proposed that the consumer
> describe something about its use of the captures, and that then the
> provider will tailor what is sent in the captures.
>
> This seems to be a change from the model that has been previously
> discussed - the three message exchange:
> capabilities/advertisement/selection. As I understand that model, the
> provider can use the capabilities from the consumer to tailor what it
> advertises to the consumer, and then the consumer selects from what has
> been advertised.
>
> ISTM that its intrinsic in the selection process that the captures that
> have been advertised have some specific semantic that is preserved when
> they are selected. This new model seems to alter that relationship. I
> don't mean to imply that it is wrong, only that it seems like a
> fundamental change.

I'll address the points relative to the consumer-layout one, partly 
because that's one I wrote and partly because I think it's the one that 
most affects this model.

This is definitely something we've thought about, however I don't 
believe it's that fundamental a change. The consumer spatial information 
(if present) doesn't place any limitations on what the provider can 
send; the provider can freely ignore it without being in any way out of 
spec and indeed in many cases (where the implementation doesn't have 
these layout concerns) should cheerfully do so. The information serves 
to help the provider make good choices about which captures to associate 
streams with so that they are rendered sensibly on the receiver.

> It also seems to me that at least part of what has been proposed in
> these drafts could be addressed by the prior model - especially since
> the content of the capabilities message is yet to be defined. For
> instance, If the consumer were to specify that it has three screens and
> a speaker with each screen, then an MCU as provider could construct an
> advertisement containing a single scene where all the entries contain
> three captures. These would presumably be alternative ways of rendering
> the conference on three screens. It wouldn't require the consumer to
> again decide its capabilities as part of the selection. I think it might
> also increase the likelihood that the MCU could reuse the same captures
> it provides for more than one endpoint.

This is a good point and would certainly work for fixed screen layouts, 
but I can't think of a particularly good way to make it work for devices 
wanting to do local composition (eg, display more than one video capture 
per screen), as that essentially means that number and layout of screens 
is dynamic and will likely depend on the provider's advertisment. 
Potentially I suppose the consumer could resend its layout after 
receiving the provider advertisment and the provider could then resend 
the advertisment with a suitable set of captures for that layout, but 
that does seem over-complex to me.

Rob

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 01 Jun 2012 15:45:33 -0400

> so this will likely be the last email I'll be able to send until

Happy anniversary. Give my regards to the Queen. :)

> This is a good point and would certainly work for fixed screen layouts,

IIUC, what you have proposed might make the capability message redundant 
- moving that info into the selection message. If that's really true it 
would eliminate one message, which would be good. But its a change.

But that wouldn't cover one of the supposed advantages of the capability 
message - that it could prune the set of things advertised to a more 
manageable number than is required to cover all possible configurations 
of consumers. If the capability message is still useful, then can't it 
eliminate the need to put comparable info in the selection?

> is dynamic and will likely depend on the provider's advertisment.

I think this may be an area that needs more exploration of use cases and 
alternative solutions.

> that does seem over-complex to me.

Yes, that would suggest that the messaging model isn't quite right.

There has been very little discussion of what the capability message 
should look like. I wonder if digging into that would help. It would be 
a matter of defining what info about the consumer would be beneficial 
for the provider in constructing advertisements.

I would imagine wanting to say something like:

- I have 3 screens with certain geometry, three speakers with certain 
geometry, and no ability to compose audio or video - only an ability to 
couple captures to particular devices. Please send me an advertisement 
that works for this.

- OR, I can handle 20 video and 20 audio captures, and am prepared to 
compose and/or switch them locally to my equipment. Please send me an 
advertisement that gives me flexibility up to this limit.

The thing I found most disconcerting in the three proposals was the 
notion that there are advertised captures whose content is not well 
defined at the time of advertisement - that will presumably be defined 
in some way after the selection is done. Something seems fundamentally 
wrong about this. If one can select varying numbers of captures of this 
sort from a scene entry, how does one decide *which* ones to select? If 
only the quantity selected matter then the mechanism seems wrong.

Perhaps there should be different sorts of capture scene entries:
- those that contain specific captures, each with well defined content.
   For these you can select individual captures.
- those that only contain a quantity of captures. For these you can
   only select the complete entry.
- others?

	Thanks,
	Paul

---

From: "Johan Ludvig Nielsen (johaniel)" <[email-masked]>
Date: Tue, 5 Jun 2012 12:21:24 +0200

About the audio part. I think we can simplify and generalize by talking
about association of an audio stream to video streams instead of
rendering positions. This should work for both fixed screens and local
composition. The consumer should never have to tell the provider about
its loudspeaker layout.

Johan

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Paul Kyzivat
Sent: 1. juni 2012 21:46
To: [email-masked]
Subject: Re: [clue] negotiation between consumer and provider

On 6/1/12 12:14 PM, Robert Hansen wrote:
> (NB, the UK is off for the next four days to celebrate our 
> queenliness, so this will likely be the last email I'll be able to 
> send until Wednesday).

>> it is that data included by the consumer in its selection message to 
>> the provider might alter in a fundamental way what content is 
>> included in the selected captures. More specifically, its proposed 
>> that the consumer describe something about its use of the captures, 
>> and that then the provider will tailor what is sent in the captures.
>>

>> advertises to the consumer, and then the consumer selects from what 
>> has been advertised.
>>
>> ISTM that its intrinsic in the selection process that the captures 
>> that have been advertised have some specific semantic that is 
>> preserved when they are selected. This new model seems to alter that 
>> relationship. I don't mean to imply that it is wrong, only that it 
>> seems like a fundamental change.
>
> because that's one I wrote and partly because I think it's the one 
> that most affects this model.
>
> believe it's that fundamental a change. The consumer spatial 
> information (if present) doesn't place any limitations on what the 
> provider can send; the provider can freely ignore it without being in 
> any way out of spec and indeed in many cases (where the implementation

> doesn't have these layout concerns) should cheerfully do so. The 
> information serves to help the provider make good choices about which 
> captures to associate streams with so that they are rendered sensibly
on the receiver.

>> instance, If the consumer were to specify that it has three screens 
>> and a speaker with each screen, then an MCU as provider could 
>> construct an advertisement containing a single scene where all the 
>> entries contain three captures. These would presumably be alternative

>> ways of rendering the conference on three screens. It wouldn't 
>> require the consumer to again decide its capabilities as part of the 
>> selection. I think it might also increase the likelihood that the MCU

>> could reuse the same captures it provides for more than one endpoint.
>

> but I can't think of a particularly good way to make it work for 
> devices wanting to do local composition (eg, display more than one 
> video capture per screen), as that essentially means that number and 
> layout of screens is dynamic and will likely depend on the provider's
advertisment.

	Thanks,
	Paul
_______________________________________________
clue mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/clue

---

From: "Espen Berger (espeberg)" <[email-masked]>
Date: Wed, 6 Jun 2012 16:41:30 +0200

I do not think we need the consumer capabilities message to achieve any
of this use cases. 

What I think is needed (as you suggest) is to work through some of the
use cases and look at possible advertisement and configuration messages
work together, to review how we can make the use cases work with whats
already in the CLUE framework.

>From a use case perspective I think the endpoint vendors should
advertise a alternatives that are possible to render on single, dual and
triple screen endpoints. It does not help interoperability if a new
vendors arrive with an advertisement of capture streams that has a
strict requirement to be rendered on oddly shaped screen setup.  

Cheers 

-Espen 

-----Original Message-----

on the receiver.

advertisment.

	Thanks,
	Paul