Subject: [clue] Updates in framework-02

From: "Duckworth, Mark" <[email-masked]>
Date: Fri, 6 Jan 2012 14:51:37 -0800

The framework-02 document has updates to the sections about describing spatial relationships, using a coordinate system and attributes for area of capture and point of capture.  This is based on the presentation and discussion at the Taipei meeting, and email list discussion.  Please review and post comments.

Mark Duckworth

---

From: "Roni Even" <[email-masked]>
Date: Tue, 10 Jan 2012 16:31:14 +0200

Hi Mrak,

Just to clarify, you are asking for review of sections 5.3 till 5.6.2

Roni

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Duckworth, Mark
Sent: Saturday, January 07, 2012 12:52 AM
To: [email-masked]
Subject: [clue] Updates in framework-02

The framework-02 document has updates to the sections about describing
spatial relationships, using a coordinate system and attributes for area of
capture and point of capture.  This is based on the presentation and
discussion at the Taipei meeting, and email list discussion.  Please review
and post comments.

Mark Duckworth

---

From: "Duckworth, Mark" <[email-masked]>
Date: Tue, 10 Jan 2012 19:43:14 -0800

Hi Roni,

The relevant parts to review are these sections:
5.3. Describing Spatial Relations
5.4.4. Area of capture
5.4.5. Point of capture
5.6. Attributes for Capture Sets
5.6.2 Area Scale Millimeters  (I think the name should be changed to just "area scale")
9. Using the Framework

This link to the differences might be useful to help you review what has changed:
http://tools.ietf.org/wg/clue/draft-ietf-clue-framework/draft-ietf-clue-framework-02-from-01.diff.html

Regards,
Mark

From: Roni Even [mailto:[email-masked]]
Sent: Tuesday, January 10, 2012 9:31 AM
To: Duckworth, Mark; [email-masked]
Subject: RE: [clue] Updates in framework-02

Hi Mrak,

From: [email-masked] [mailto:[email-masked]] On Behalf Of Duckworth, Mark
Sent: Saturday, January 07, 2012 12:52 AM
To: [email-masked]
Subject: [clue] Updates in framework-02

Mark Duckworth

---

From: "Roni Even" <[email-masked]>
Date: Fri, 13 Jan 2012 20:19:12 +0200

Hi,

I read the draft again till section 6 in order to review the parts you
mentioned. It is better than 01 but I still have some comments.  

I will list all of them in this email and afterwards we can cut out the ones
that remain open if any? Some of them are editorial. I will comment on all
section 5.

1 The second sentence in section 5 says "figure" no number and no number to
the figure.

2. The figure in section 5 need to show that capture set is composed of rows
and each row is made from media captures.

3. The second sentence in 5.1 is not needed you just need to use the term
and not repeat part of the definition.

4. We need a term for the "row of media captures inside a capture set"

5. In 5.1 fourth paragraph  change to "Media Captures may also be associated
with data constructs that capture  encoding aspects of the streams" .

6.  In 5.2 fourth paragraph suggest adding "may be"  to  become "A media
capture is described by Attributes and may be associated with an  Encode
Group, and Simultaneous Transmission Set."

7. In 5.3 first paragraph I suggest changing to "Within a capture set, a
coordinate system should be used to  model relative or absolute positions of
its constituent media

   captures.  This can allow precise physical sizes to be determined if
required, as well as the order in which media captures that form a   group
should be rendered. "

The co-ordinate system is needed if you want to describe the spatial
relations. I also changed the "displayed" to "rendered"

8. In 5.3 second bullet suggest to change to "for example it shall provide
co-ordinate information for 3 side-by-side participant .". . it does not
matter if absolute or relative or no scale but it is needed to describe the
spatial relations.

9. In 5.3 in the third bullet, No physical scale means no spatial relation
or is it to provide some way to describe the rendering order (e.g. for MCU
case). 

10. Audience front at the end of section 5.3 is not specified, there is
front and back in section 3

11. In section 5.4 first paragraph what do you mean by static, Is media
capture describing a static aspect of a stream? My view is that attribute
are not static but may define some range, like maximum value or a model for
switching.

12. In 5.4 second paragraph you mention "most common use case" what are
those, maybe these is why we are discussing what attributes and values
should be specified here.

13. in 5.4.1 I am not sure what main is and what should not be main, I can
understand the logic in RFC 4796. For example can we specify "speaker" as a
purpose.  

As for main I can see a media capture that shows the whole room as main yet
a VC1 that shows left side of a room is not main, I will see as main a "row"
that has (VC1 - left, VC2-middle, VC3-right). So should we have VC1 with
purpose main even when stands by itself.

14. On 5.4.2 we are having a separate discussion about what should be
defined in this document.

15. in 5.4.4 first paragraph "By comparing the area of   capture for
different media captures, " I think you need to add of the same capture
scene or capture set.

16. in 5.4.4 third paragraph you say that the planes should intersect, why
is that, I think that there can be a gap between the planes at the area of
capture.  The area of captures are in the area of scene defined later.

17. In 5.4.4 you mention 4 points but not the order they are listed, in the
example in section 9 you give a table but the headers are not part of the
definition, maybe specify from top left clockwise direction.

18. Suggest to change the start of the last paragraph in 5.4.4 to "If the
area of capture attribute is specified, it must include X, Y, Z .". I do not
like the use of "optional" in the current sentence.

19. Section 5.4.5 suggest to change the beginning to " The point_of_capture
attribute is used to describe the location  of a capture device or
pseudo-device. A use case is if there are multiple captures ."

20. We have a separate open issue on 5.4.6.

21. In 5.5 the first two paragraph use capture set and capture scene, we
need to clarify the relation between them, maybe in section3. We discussed
it in Taipei. For example are the two capture sets in the example in the
second paragraph one or two capture scenes.

22. in 5.5 ""Media Captures of the same media type are associated with each
other  by grouping them together in a single row of a Capture Set" .  I
think that the grouping in a single row means simultaneous, does the
grouping imply any other association without using any attribute? If not
maybe say " simultaneous"

23. In 5.5 " Association between audio and video can be made by finding
audio and   video captures which share overlapping areas of capture." I
think this requires that they will use the same origin, I did not see it
required.

24. In 5.5 " A media consumer could choose one row of each media type (e.g.,
audio  and video) from a capture set.  For example a three stream consumer
could choose the first video row plus the audio row, while a single

   stream consumer could choose the second or third video row plus the
audio row.  An MCU consumer might choose to receive multiple rows."  A
single stream consumer may also choose VC0 or VC1 or VC2, maybe clarify it
since it is the non obvious case.

25. I think that the last paragraph in section 5.5 should be removed and
discussed when you define the Simultaneous Transmission Sets and Encoding
Groups, it is out of context here and this is why I cannot even discuss it
when talking about this section.

26. 5.6 talks about attributes of capture set while the first bullet is
about capture scene. This is again about the lack of clarity about these two
terms.

27. In 5.6.1 I suggest adding the following sentence in the end "The area of
captures specified for this capture set must use the same co-ordinate system
(same origin)"

This is the end of my long list

Roni Even 

 

 

 

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Duckworth, Mark
Sent: Wednesday, January 11, 2012 5:43 AM
To: [email-masked]
Subject: Re: [clue] Updates in framework-02

Hi Roni,

5.4.4. Area of capture

5.4.5. Point of capture

5.6.2 Area Scale Millimeters  (I think the name should be changed to just
"area scale")

9. Using the Framework

This link to the differences might be useful to help you review what has
changed:

http://tools.ietf.org/wg/clue/draft-ietf-clue-framework/draft-ietf-clue-fram
ework-02-from-01.diff.html

Regards,

Mark

  _____

---

From: "Duckworth, Mark" <[email-masked]>
Date: Mon, 16 Jan 2012 16:00:52 -0800

Hi Roni,

Thanks for the suggestions, this is very useful.  We will use these for the next version, which is in progress now.

I have a few questions and comments on some of your items.  And I answered some of your questions.  If an item is not listed here, it means I agree with it.

3. The second sentence in 5.1 is not needed you just need to use the term and not repeat part of the definition.

Markd> I assume you mean the second sentence in the second paragraph, correct? "As  defined, a capture scene is the source scene that is captured by media devices"?

Markd> How about "Capture Set View"?  I agree a more descriptive term rather than "row" would be good.

5. In 5.1 fourth paragraph  change to "Media Captures may also be associated with data constructs that capture  encoding aspects of the streams" .

6.  In 5.2 fourth paragraph suggest adding "may be"  to  become "A media capture is described by Attributes and may be associated with an  Encode Group, and Simultaneous Transmission Set."

Markd> I think these need more discussion, about what is optional and what is required regarding the encode groups.  It sounds like you would like it to be optional, while the authors are thinking it is mandatory.

9. In 5.3 in the third bullet, No physical scale means no spatial relation or is it to provide some way to describe the rendering order (e.g. for MCU case).

Markd> We can clarify this.  Even with no physical scale, it still can represent a spatial relation between media captures.

11. In section 5.4 first paragraph what do you mean by static, Is media capture describing a static aspect of a stream? My view is that attribute are not static but may define some range, like maximum value or a model for switching.

Markd> Good question, I've been questioning this myself.  I don't see any attribute where a maximum value makes sense.  Is a model for switching still a static attribute value?  I think it can be.  One way to look at it is this - can an attribute value change "on the fly" without a new message exchange between consumer and producer, with producer advertising the new attribute values and the consumer then making a choice to receive some streams?  I have been thinking no, the attribute values can not change without this exchange.  I would call this static, to an extent, but maybe that is not a good word to use here.

12. In 5.4 second paragraph you mention "most common use case" what are those, maybe these is why we are discussing what attributes and values should be specified here.

Markd> Good point.  Does the framework support all the use cases in the use case document?  I'm not sure.

13. in 5.4.1 I am not sure what main is and what should not be main, I can understand the logic in RFC 4796. For example can we specify "speaker" as a purpose.

As for main I can see a media capture that shows the whole room as main yet a VC1 that shows left side of a room is not main, I will see as main a "row" that has (VC1 - left, VC2-middle, VC3-right). So should we have VC1 with purpose main even when stands by itself.

Markd> Yes, VC1 is intended to be "main" in this case.  A "row" does not have attributes (although people have mentioned maybe there should be attributes for a "row").  Maybe we need some additional values for purpose?  Can there be more than one value for a particular media capture?

16. in 5.4.4 third paragraph you say that the planes should intersect, why is that, I think that there can be a gap between the planes at the area of capture.  The area of captures are in the area of scene defined later.

Markd> We can clarify this.  It means the infinite planes should intersect, but the areas of capture (quadrilaterals in the planes) may or may not intersect.

21. In 5.5 the first two paragraph use capture set and capture scene, we need to clarify the relation between them, maybe in section3. We discussed it in Taipei. For example are the two capture sets in the example in the second paragraph one or two capture scenes.

Markd> Different scenes.  I suggest there is a one to one correspondence between a capture scene and a capture set.  That's how I've been thinking of it.

22. in 5.5 ""Media Captures of the same media type are associated with each other  by grouping them together in a single row of a Capture Set" .  I think that the grouping in a single row means simultaneous, does the grouping imply any other association without using any attribute? If not maybe say " simultaneous"

Markd> Yes it means simultaneous.  It is also a suggestion from the provider, intending to make the consumers job of selecting captures to receive easier, if it wants to follow the suggestion.

Mark

From: Roni Even [mailto:[email-masked]]
Sent: Friday, January 13, 2012 1:19 PM
To: Duckworth, Mark; [email-masked]
Subject: RE: [clue] Updates in framework-02 - section 5 review.

Hi,
I read the draft again till section 6 in order to review the parts you mentioned. It is better than 01 but I still have some comments.
I will list all of them in this email and afterwards we can cut out the ones that remain open if any? Some of them are editorial. I will comment on all section 5.

1 The second sentence in section 5 says "figure" no number and no number to the figure.

2. The figure in section 5 need to show that capture set is composed of rows and each row is made from media captures.

3. The second sentence in 5.1 is not needed you just need to use the term and not repeat part of the definition.

5. In 5.1 fourth paragraph  change to "Media Captures may also be associated with data constructs that capture  encoding aspects of the streams" .

6.  In 5.2 fourth paragraph suggest adding "may be"  to  become "A media capture is described by Attributes and may be associated with an  Encode Group, and Simultaneous Transmission Set."

7. In 5.3 first paragraph I suggest changing to "Within a capture set, a coordinate system should be used to  model relative or absolute positions of its constituent media

   captures.  This can allow precise physical sizes to be determined if  required, as well as the order in which media captures that form a   group should be rendered. "

The co-ordinate system is needed if you want to describe the spatial relations. I also changed the "displayed" to "rendered"

8. In 5.3 second bullet suggest to change to "for example it shall provide  co-ordinate information for 3 side-by-side participant ...". . it does not matter if absolute or relative or no scale but it is needed to describe the spatial relations.

9. In 5.3 in the third bullet, No physical scale means no spatial relation or is it to provide some way to describe the rendering order (e.g. for MCU case).

10. Audience front at the end of section 5.3 is not specified, there is front and back in section 3

11. In section 5.4 first paragraph what do you mean by static, Is media capture describing a static aspect of a stream? My view is that attribute are not static but may define some range, like maximum value or a model for switching.

12. In 5.4 second paragraph you mention "most common use case" what are those, maybe these is why we are discussing what attributes and values should be specified here.

13. in 5.4.1 I am not sure what main is and what should not be main, I can understand the logic in RFC 4796. For example can we specify "speaker" as a purpose.

As for main I can see a media capture that shows the whole room as main yet a VC1 that shows left side of a room is not main, I will see as main a "row" that has (VC1 - left, VC2-middle, VC3-right). So should we have VC1 with purpose main even when stands by itself.

14. On 5.4.2 we are having a separate discussion about what should be defined in this document.

15. in 5.4.4 first paragraph "By comparing the area of   capture for different media captures, " I think you need to add of the same capture scene or capture set.

16. in 5.4.4 third paragraph you say that the planes should intersect, why is that, I think that there can be a gap between the planes at the area of capture.  The area of captures are in the area of scene defined later.

17. In 5.4.4 you mention 4 points but not the order they are listed, in the example in section 9 you give a table but the headers are not part of the definition, maybe specify from top left clockwise direction.

18. Suggest to change the start of the last paragraph in 5.4.4 to "If the area of capture attribute is specified, it must include X, Y, Z ...". I do not like the use of "optional" in the current sentence.

19. Section 5.4.5 suggest to change the beginning to " The point_of_capture attribute is used to describe the location  of a capture device or pseudo-device. A use case is if there are multiple captures ..."

21. In 5.5 the first two paragraph use capture set and capture scene, we need to clarify the relation between them, maybe in section3. We discussed it in Taipei. For example are the two capture sets in the example in the second paragraph one or two capture scenes.

22. in 5.5 ""Media Captures of the same media type are associated with each other  by grouping them together in a single row of a Capture Set" .  I think that the grouping in a single row means simultaneous, does the grouping imply any other association without using any attribute? If not maybe say " simultaneous"

23. In 5.5 " Association between audio and video can be made by finding audio and   video captures which share overlapping areas of capture." I think this requires that they will use the same origin, I did not see it required.

24. In 5.5 " A media consumer could choose one row of each media type (e.g., audio  and video) from a capture set.  For example a three stream consumer  could choose the first video row plus the audio row, while a single

   stream consumer could choose the second or third video row plus the  audio row.  An MCU consumer might choose to receive multiple rows."  A single stream consumer may also choose VC0 or VC1 or VC2, maybe clarify it since it is the non obvious case.

25. I think that the last paragraph in section 5.5 should be removed and discussed when you define the Simultaneous Transmission Sets and Encoding Groups, it is out of context here and this is why I cannot even discuss it when talking about this section.

26. 5.6 talks about attributes of capture set while the first bullet is about capture scene. This is again about the lack of clarity about these two terms.

27. In 5.6.1 I suggest adding the following sentence in the end "The area of captures specified for this capture set must use the same co-ordinate system (same origin)"

Roni Even

Hi Roni,

Regards,
Mark

---

From: "Roni Even" <[email-masked]>
Date: Tue, 17 Jan 2012 10:37:34 +0200

Hi Mark,

Thanks, I assume by agreeing they will be addressed in the next revision for
the editorial ones.

Inline for your questions

Roni

From: Duckworth, Mark [mailto:[email-masked]] 
Sent: Tuesday, January 17, 2012 2:01 AM
To: Roni Even; [email-masked]
Subject: RE: [clue] Updates in framework-02 - section 5 review.

Hi Roni,

Thanks for the suggestions, this is very useful.  We will use these for the
next version, which is in progress now.

I have a few questions and comments on some of your items.  And I answered
some of your questions.  If an item is not listed here, it means I agree
with it.

Markd> I assume you mean the second sentence in the second paragraph,
correct? "As  defined, a capture scene is the source scene that is captured
by media devices"?

Roni>Yes , the one starting with "as defined ."

Markd> How about "Capture Set View"?  I agree a more descriptive term rather
than "row" would be good.

Roni> The name sounds OK but it will make sense to define it in the
definition section.

Markd> I think these need more discussion, about what is optional and what
is required regarding the encode groups.  It sounds like you would like it
to be optional, while the authors are thinking it is mandatory.

Roni> I can start a thread on encoding groups while I think that
simultaneous transmission sets are optional since the provider may send just
one row like {vc1, vc2,vc3}.

Looking at this section again maybe it will make sense to divide it to two
sections one about the model description which references also the other
sections of the document and  one that defines the attributes.

Markd> We can clarify this.  Even with no physical scale, it still can
represent a spatial relation between media captures.

Roni> I am not sure,  in 5.6 for area of scale we have physical scale which
provide real units like mm,  unknown which provides a scale but is not
physical just numbers that can be used to understand the scale and  no
scale.

On the no scale I see two types. The first is that there is some spatial
order and the renderer should display them accordingly, example is MCU
wanting to enforce which stream will go to which monitor at the receiver
even if they represent different rooms, the other is where the provider do
not want to enforce any specific render order like an MCU sending three
streams each from a different system but does not care for the render order.

Maybe we need four options and not three.

Markd> Good question, I've been questioning this myself.  I don't see any
attribute where a maximum value makes sense.  Is a model for switching still
a static attribute value?  I think it can be.  One way to look at it is this
- can an attribute value change "on the fly" without a new message exchange
between consumer and producer, with producer advertising the new attribute
values and the consumer then making a choice to receive some streams?  I
have been thinking no, the attribute values can not change without this
exchange.  I would call this static, to an extent, but maybe that is not a
good word to use here.

Roni> I think that attribute values are static in a sense that to change the
current value it need to be negotiated. Now the value itself can specify a
range but this should be part of the value definition. I think we need to
clarify what is meant by static (value and not attribute)

Markd> Good point.  Does the framework support all the use cases in the use
case document?  I'm not sure.

Roni>maybe just as an editorial change delete this part and start the
sentence with the fact that new attribute can be added based on use cases.

Markd> Yes, VC1 is intended to be "main" in this case.  A "row" does not
have attributes (although people have mentioned maybe there should be
attributes for a "row").  Maybe we need some additional values for purpose?
Can there be more than one value for a particular media capture?

Roni> I am not sure that in the case of a row {VC1, VC2, VC3} I will have
VC1 as main always. Since my understanding is that a consumer can decide it
wants to take only one of the captures in the row it may make sense for the
provider to say for example that VC1 is more important than the other two.
I think we need more values initially and this is why I referenced RFC 4796.

Markd> We can clarify this.  It means the infinite planes should intersect,
but the areas of capture (quadrilaterals in the planes) may or may not
intersect.

Roni> This is what I meant.

Markd> Different scenes.  I suggest there is a one to one correspondence
between a capture scene and a capture set.  That's how I've been thinking of
it.

Roni> We should have it as an open issue that needs to be clarified since I
heard other opinions. 

Markd> Yes it means simultaneous.  It is also a suggestion from the
provider, intending to make the consumers job of selecting captures to
receive easier, if it wants to follow the suggestion.

Roni> This was an editorial comment that if associated means simultaneous
than it should say so. In the past it also meant spatial order but we are
now using explicit spatial relation which means according to my
understanding that {VC1, VC2, VC3} is the same as {VC2, VC1, VC3}

Mark

Hi,

 

Roni Even 

 

 

 

 

Hi Roni,

5.4.4. Area of capture

5.4.5. Point of capture

9. Using the Framework

Regards,

Mark