Subject: [clue] draft-ietf-clue-data-model-schema-00

From: Paul Kyzivat <[email-masked]>
Date: Mon, 22 Jul 2013 11:41:19 -0400

A couple of things I picked up while reviewing this version:

Section 3 & Section 22:

The schema was updated so captureEncodingType references 
encodingParametersType, but there is a typo (cut/paste error I guess) so 
that the intended definition of encodingParametersType actually 
redefines captureParametersType.

Section 10.7 <priority>:

    ... The higher the
    importance, the lower the contained value.  When media captures are
    marked with a "0" priority value, it means that they are "not subject
    to priority".

    [edt's note: discussion needed]

IMO it makes no sense for some captures to be subject to priority and 
others to not be subject to priority. What would that mean? The only 
thing I can see is that those are *mandatory* to render. But we can't 
really mandate that. If you can't do it we can't force you. So the best 
it could mean is "really really important" to render. But that is the 
same as having the highest priority.

So I suggest that zero is just the highest priority - no more, no less.

And I suggest we also make zero the *default* priority, so that all 
captures have one, implicitly or explicitly. Then, if you don't mention 
priority at all, then all captures are equal. And you can just add 
priority for those captures that are less important.

Equivalently, we could leave priority optional, and specify that the 
absence of a priority implicitly means the highest priority. Then the 
presence of priority means a lower priority. But I think this 
formulation is a little more confusing to explain.

Section 10.10. <mobility>:

    <mobility> is an optional element indicating wheter or not the
    capture device originating the capture moves during the telepresence
    session.  ...

We can't know for sure that it will move, only that it may. Also a typo. So:

s/moves/may move/
s/wheter/whether/

	Thanks,
	Paul

---

From: Christian Groves <[email-masked]>
Date: Tue, 23 Jul 2013 10:41:24 +1000

Hello Paul,

I agree that a default is needed.

The reason for a "no priority" I saw a case where an Advertiser for 
example may offer prioritised audio but choose not to prioritise video 
or vice versa. Rather than prioritise audio over video or vice versa 
which would effectively happen if all captures had a default priority level.

Regards, Christian

On 23/07/2013 1:41 AM, Paul Kyzivat wrote:
> encodingParametersType, but there is a typo (cut/paste error I guess) 
> so that the intended definition of encodingParametersType actually 
> really mandate that. If you can't do it we can't force you. So the 
> best it could mean is "really really important" to render. But that is 
> the same as having the highest priority.
>
> captures have one, implicitly or explicitly. Then, if you don't 
> mention priority at all, then all captures are equal. And you can just 
> add priority for those captures that are less important.
>
> _______________________________________________
> clue mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/clue
>

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 22 Jul 2013 22:48:09 -0400

> which would effectively happen if all captures had a default priority

Audio and video don't "compete" with one another. I can't see how the 
priority processing of one would interact with the other.

Perhaps we need to say something that audio and video are independently 
prioritized.

	Thanks,
	Paul

---

From: Christian Groves <[email-masked]>
Date: Tue, 23 Jul 2013 14:00:04 +1000

Hello Paul,

"Audio and video don't "compete" with one another." Currently priorities 
can be set across all capture types. E.g. I can say that's more 
important for you to get an audio stream and a presentation stream than 
it is to get the video if you have to choose because of some 
bandwidth/display/etc contraint. It wouldn't be possible if you consider 
priority to be scoped to a single media.

Regards, Christian

On 23/07/2013 12:48 PM, Paul Kyzivat wrote:
> Perhaps we need to say something that audio and video are 
> independently prioritized.
>
>     Thanks,
>     Paul
>
>> Regards, Christian
>>

---

From: Roberta Presta <[email-masked]>
Date: Tue, 23 Jul 2013 13:39:35 +0200

Hi all,

another possibility is the following.
Priority is an optional attribute.
When it is absent, it is intended to be "the lowest priority".
When it is present, you have to order the priority values in a way that 
the most important capture has the lowest numeric value.

For example, if I have
- the audio of the lecturer AC0,
- the video of the lecturer VC0,
- the video of the slides VC1,
- the video of the overall room VC2
and I want to prioritize the audio of the lecturer, and then the video 
of the slides, I can set the priority values as follows:
- AC0, priority = 1
- VC1, priority = 2
- other captures, no priority

The media consumer will know that:
- the most important capture is AC0
- the second one is VC1
- all the other captures have the same level of importance, which is 
lower than the one of  AC0 and VC1.

What is your feeling about it?
Cheers,

Roberta

Il 23/07/2013 06:00, Christian Groves ha scritto:
> Hello Paul,
>
> "Audio and video don't "compete" with one another." Currently 
> priorities can be set across all capture types. E.g. I can say that's 
> more important for you to get an audio stream and a presentation 
> stream than it is to get the video if you have to choose because of 
> some bandwidth/display/etc contraint. It wouldn't be possible if you 
> consider priority to be scoped to a single media.
>
> Regards, Christian
>

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 25 Jul 2013 20:48:19 +0000

Hi,

Even if the advertiser thinks something has priority, the consumer may have another opinion.

Isn't the role/content information supposed to describe what content is associated with the capture, and then the consumer chooses what it wants - based on its own priorities?

Regards,

Christer

Sent from Windows using TouchDown (www.nitrodesk.com)

-----Original Message-----
From: Roberta Presta [[email-masked]]
To: [email-masked] [[email-masked]]
Subject: Re: [clue] draft-ietf-clue-data-model-schema-00
Hi all,

For example, if I have

Roberta

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 25 Jul 2013 18:14:37 -0400

> What is your feeling about it?

It "works". The key advantage is that it provides a concise way to 
specify "lowest priority". Without this, the lowest priority could be 
specified as the maximum allowable priority value. (Is there an upper 
bound for unsigned int? 2^32-1?)

Implementations are going to need a representation for all values, 
including the highest and lowest. If we are to represent priorities as 
unsigned integers, then I think both lowest and highest should fall into 
that range so that the implementation can use an unsigned integer too. 
So *if* we want the default to be lowest priority, then I would vote 
that we make the default be the highest value representable in the 
range, rather than something lower than that.

I guess the question is whether we want the default to be highest 
priority or lowest priority.

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 25 Jul 2013 18:16:49 -0400

On 7/25/13 4:48 PM, Christer Holmberg wrote:
> Isn't the role/content information supposed to describe what content is
> associated with the capture, and then the consumer chooses what it wants
> - based on its own priorities?

Its another attribute that the consumer can consider (or ignore).

	Thanks
	Paul

> Regards,
> Christer
>
>
> Sent from */Windows/* using *TouchDown* (www.nitrodesk.com)
>
> *From:* Roberta Presta [[email-masked]]
> *To:* [email-masked] [[email-masked]]
> *Subject:* Re: [clue] draft-ietf-clue-data-model-schema-00
> Hi all,
>

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 25 Jul 2013 18:23:57 -0400

> priority to be scoped to a single media.

OK. So they do compete with one another for bandwidth. So maybe if that 
is a constraint you may want to take that into account.

But isn't video bw usually so much higher than audio bw that audio bw is 
insignificant to the decisions?

If we think video and audio should be sorted together for prioritization 
then it is going to complicate assigning priorities.

As I commented to mark, how to select captures is starting to look very 
complex. We also have the problem of how to select audio captures that 
"go with" particular video captures. I guess priorities might be one way 
to do that. Some sort of spatial information might be another.

	Thanks
	Paul

---

From: Christian Groves <[email-masked]>
Date: Fri, 26 Jul 2013 15:12:51 +1000

Hello Roberta and Paul,

Please see my comments below.

Regards, Christian

>> the most important capture has the lowest numeric value.
[CNG] What do you mean by "is present"? If an endpoint needs to set the 
priority on one capture does this mean it is forced to set the priority 
on all the captures?

>> What is your feeling about it?
[CNG] It makes sense. I think the test is how the logic works for 
multiple captures for each media type. I've thought about that aspect an 
I couldn't see an issue. I think you have to assume that the consumer 
will not use the priority exclusively to determine which capture it wants.

> bound for unsigned int? 2^32-1?)
[CNG] Perhaps we should just make this a btye 0-255 possible values? And 
perhaps we need to say there is no inference to how much more important 
the priority based on the position. e.g. 1 is not 100x times more 
important than 100. If you have two captures one with priority 1 and the 
other priority 100 the priority one capture is simply more important. No 
difference to 1 & 2, 1 & 50 etc.

> including the highest and lowest. 
[CNG] Strictly speaking we probably only need the "highest" priority. 
The lowest is effectively the number before or after (depending on 
whether 1 is highest or not) those that have been set in the captures.

> If we are to represent priorities as unsigned integers, then I think 
> both lowest and highest should fall into that range so that the 
> implementation can use an unsigned integer too. So *if* we want the 
> default to be lowest priority, then I would vote that we make the 
> default be the highest value representable in the range, rather than 
> something lower than that.
>
[CNG] I think the "lowest priority".

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 26 Jul 2013 08:41:49 +0000

Hi,

We can define all kind of attributes, but what case is this needed for?

Regards,

Christer

-----Original Message-----
From: Paul Kyzivat [[email-masked]]
To: [email-masked] [[email-masked]]
Subject: Re: [clue] draft-ietf-clue-data-model-schema-00

        Thanks
        Paul

> Regards,
> Christer
>
>
> Sent from */Windows/* using *TouchDown* (www.nitrodesk.com<http://www.nitrodesk.com>)
>

---

From: Roberta Presta <[email-masked]>
Date: Fri, 26 Jul 2013 11:17:32 +0200

Hi Paul,

Il 26/07/2013 00:23, Paul Kyzivat ha scritto:
>  We also have the problem of how to select audio captures that "go 
> with" particular video captures. I guess priorities might be one way 

I think a mechanism to associate a capture to another one is needed or 
at least it is worth to be discussed.

We have currently two mechanisms to group captures semantically:
1) putting them into the same capture scene
2) using the <relatedTo> tag to indicate that a capture is linked to 
another one.

The first one is coarse-grained, while the second one has been conceived 
for linking "secondary" information, such as subtitles or translations, 
to a "primary" capture, like for example the lecturer's audio.
However, we could exploit such element to link an audio capture to a 
video capture. For example you could have the audio of the presenter - 
AC0, and the video of the presenter is marked with a <relatedTo> field 
containing AC0, and/or viceversa. Alternatively, we can design a 
specialized field to do the task.

Capture scene entries have been designed to group captures of the same 
media type, so we can not use them to answer that need.
How do you think priority and spatial information could be used to 
couple captures together?
For example, if we force captures that go together to have the same 
priority, we are "overloading" the <priority> meaning and probably 
making the priority settings (and its interpretation) even more complex.

Cheers,

Roberta

---

From: Roberta Presta <[email-masked]>
Date: Fri, 26 Jul 2013 12:35:23 +0200

Hi Christian,

please see inline.

Il 26/07/2013 07:12, Christian Groves ha scritto:
> [CNG] What do you mean by "is present"? If an endpoint needs to set 
> the priority on one capture does this mean it is forced to set the 
> priority on all the captures?
>
[RP] No, the media provider is not forced to set the priority on all the 
captures.
The media provider can mark the captures that, according to its point of 
view, are the most relevant ones. The other captures are left without 
the <priority> element - it means that they are not carrying the main 
information, from the provider perspective. By this way the media 
provider is making a suggestion to the media consumer.

> multiple captures for each media type. I've thought about that aspect 
> an I couldn't see an issue. I think you have to assume that the 
> consumer will not use the priority exclusively to determine which 
> capture it wants.
>
[RP] I totally agree on the fact that the consumer will not use 
exclusively <priority> to select the captures it needs.
>> bound for unsigned int? 2^32-1?)
[RP] Yes, that is the upper bound.
> [CNG] Perhaps we should just make this a btye 0-255 possible values? 
[RP] We can use "unsignedByte" as xs:type in that case.
> And perhaps we need to say there is no inference to how much more 
> important the priority based on the position. e.g. 1 is not 100x times 
> more important than 100. If you have two captures one with priority 1 
> and the other priority 100 the priority one capture is simply more 
> important. No difference to 1 & 2, 1 & 50 etc.
>
[RP] +1.
> whether 1 is highest or not) those that have been set in the captures.
[RP]
The values (highest and lowest) should be defined both.

> [CNG] I think the "lowest priority".
[RP]
I agree with Paul, we have to arrange the priority interpretation, 
otherwise talking about the lowest priority and the highest priority is 
misleading.
What is the clearest definition between the following alternatives?
"the highest <priority> value  = the highest relevance score (e.g. 255)"
or
"the highest <priority> value = the first relevance rank (e.g. 1)"
?

Thanks,

Roberta

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 26 Jul 2013 11:03:03 -0400

> difference to 1 & 2, 1 & 50 etc.

Why would we want to restrict to 256 values? To save 3 bytes in the 
implementation?

While people can probably settle with that, why?

> whether 1 is highest or not) those that have been set in the captures.

The highest is zero, so that is easy. The lowest you can specify is 
currently 2^32-1. Suppose somebody decides to set some captures to 
2^32-1, and leaves some unspecified? What value does the implementation 
give to them?

So just say the the default is 2^32-1, or maxint. XML allows you to put 
a default in the schema. So we can just use that.

This is assuming we want the default to be "lowest priority". If the 
default is "highest priority" then of course it is zero.

> [CNG] I think the "lowest priority".

I don't much care. But I think it should be based on what we think the 
most common case is. Will people want to just mark a few captures as 
lower than the rest? Or just mark a few higher than the rest?

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 26 Jul 2013 11:04:21 -0400

> We can define all kind of attributes, but what case is this needed for?

We've had this one for awhile. We are just discussing the details.
IMO its just a way for the advertiser to make recommendations.

	Thanks,
	Paul

> *From:* Paul Kyzivat [[email-masked]]
>> Sent from */Windows/* using *TouchDown* (www.nitrodesk.com <http://www.nitrodesk.com>)
>>

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 26 Jul 2013 11:07:54 -0400

> at least it is worth to be discussed.

+1

> specialized field to do the task.

Good idea. <relatedTo> seems to do the job.

> couple captures together?

Well, there has been some discussion that spatial info could be used to 
tie an audio capture to a video capture. But John has debunked that idea.

> making the priority settings (and its interpretation) even more complex.

I agree.

	Thanks,
	Paul

---

From: Christian Groves <[email-masked]>
Date: Mon, 29 Jul 2013 10:45:57 +1000

Hello Paul,

Please see below.

Regards, Christian

> While people can probably settle with that, why?

[CNG] Its not to save the 3 bytes in signalling. I just think it is 
logically better to give a finite set of priorities within the realms of 
day to day experience. If I was to set this via some UI I think its 
easier present a range 0 - 256 rather than 0-2,147,483,647.

> 2^32-1, and leaves some unspecified? What value does the 
> implementation give to them?
>
> So just say the the default is 2^32-1, or maxint. XML allows you to 
> put a default in the schema. So we can just use that.
>

[CNG] I'm OK with Roberta's proposal:
"the highest <priority> value = the first relevance rank (e.g. 1)"
default lowest value 255.

---

From: Christian Groves <[email-masked]>
Date: Mon, 29 Jul 2013 11:10:20 +1000

Hello Roberta,

Regards, Christian

> Good idea. <relatedTo> seems to do the job.
[CNG] It could be made to work but it might be clearer to have a 
separate attribute with similar syntax. The original intention behind 
the "relatedTo" was that the capture provided "additional or 
complementary information" to another capture. e.g. There's a main audio 
capture AC1. I have a capture which is a commentary of the conference 
AC2(relatedTo(AC1)) [btw. as current defined in the framework there's 
only one value for the relatedTo capture]. Semantically this is 
different from "this capture is an audio representation of these video 
captures".
> Well, there has been some discussion that spatial info could be used 
> to tie an audio capture to a video capture. But John has debunked that 
> idea.
[CNG] As Paul mentioned it seems the current point is that audio 
captures won't have the same spatial information as video so if won't be 
possible to tie audio and video through spatial information. That's 
where the <relatedTo> will probably become more important.
>> making the priority settings (and its interpretation) even more complex.
[CNG] I don't think we want to overload priority.

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 29 Jul 2013 23:25:59 +0200

> easier present a range 0 - 256 rather than 0-2,147,483,647.

I don't imagine a UI for setting these, at least not for regular people. 
Do you? Its more conceivable that a UI for users might *show* 
priorities. But then you only show the ones that were used.

When I was looking at one example, I was thinking I would like to use 
values like:
- 100, 101, 102, ... for one CSE or scene
- 200, 201, 202, ... for a different CSE or scene

One can easily use up more than 256 that way.

Another possibility is to use real numbers in range [0,1].

Then I can use 0.1, 0.2, ... or 0.10, 0.11, 0.12,... 0.20, 0.21, ...
and can in general stick something between any two I have previously 
defined.

	Thanks,
	Paul

---

From: Christian Groves <[email-masked]>
Date: Tue, 30 Jul 2013 09:53:18 +1000

Hello Paul,

I'm thinking that this could be set by whoever is configuring the system 
at installation or at the start of a conference. Not something that is 
hard coded. So that's why I thought a more "user friendly" range would 
be good. Utilising 100, 101, 102, 200, 201, 202 etc... I think is more 
of a programmer's view on the world. I do the same thing when writing 
g-code for my mill but I'm not sure its a practise for telepresence 
endpoints.

It would be good to get other views on this.

Regards, Christian

On 30/07/2013 7:25 AM, Paul Kyzivat wrote:
> I don't imagine a UI for setting these, at least not for regular 
> people. Do you? Its more conceivable that a UI for users might *show*

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 30 Jul 2013 07:56:10 +0200

> g-code for my mill but I'm not sure its a practise for telepresence

If the goal is to make the *range* "user friendly" for non-programmers, 
then it ought to be some number of decimal digits rather than powers of two.

> It would be good to get other views on this.

Yep.

	Thanks,
	Paul