Subject: [clue] again on <spatialInformation>

From: Roberta Presta <[email-masked]>
Date: Fri, 26 Oct 2012 12:13:22 +0200

Hi all,

we have tried to restructure the data model format according to the 
following considerations:
- we are interested in spatial relationship among media captures 
belonging to the same capture scene
- we are not interested in spatial relationship among media captures 
belonging to different capture scenes
- each capture scene has an independent coordinate space (different 
capture scenes can have different coordinate spaces)
- coordinates within the same scene must share the same unit of measure 
and coordinate space

Given these assumptions, spatial information associated with media 
captures makes sense only within the scope of the same capture scene.
Spatial information of media captures must be then associated with a 
certain capture scene, in order to be spatially related.
We have reintroduced a <spatialInformation> element in the media capture 
type with a mandatory attribute, "sceneIDREF", containing the ID of the 
scene the media capture is associated with.
The <spatialInformation> element appears only in "spatially definible" 
media captures (i.e., captures that can be spatially described) and 
provides information about capture points, capture axis points and 
capture areas.

Below you please find the datamodel updates and further comments:

      <!-- MEDIA CAPTURE TYPE -->
      <xs:complexType name="mediaCaptureType" abstract="true">
         <xs:sequence>
          <xs:element ref="description" minOccurs="0"/>
          <xs:element name="capturedMedia" type="xs:string" minOccurs="0"/>
          <xs:element name="encGroupIDREF" type="xs:IDREF"/>
          <xs:element name="content" type="xs:string" minOccurs="0"/>
          <xs:element name="switched" type="xs:boolean" minOccurs="0"/>
          <xs:element name="composed" type="xs:boolean" minOccurs="0"/>
          <xs:element name="maxCaptureEncodings" type="xs:unsignedInt" 
minOccurs="0"/>
          <xs:choice>
              <xs:sequence>
               <xs:element name="spatialInformation" 
type="tns:spatialInformationType" maxOccurs="unbounded"/>
             </xs:sequence>
              <xs:element name="nonSpatiallyDefinible" type="xs:boolean" 
fixed="true"/>
</xs:choice>
          <xs:any namespace="##other" processContents="lax" 
minOccurs="0" maxOccurs="unbounded"/>
         </xs:sequence>
         <xs:attribute name="captureID" type="xs:ID" use="required"/>
         <xs:anyAttribute namespace="##other" processContents="lax"/>
      </xs:complexType>

      <!-- SPATIAL INFORMATION TYPE -->
      <xs:complexType name="spatialInformationType">
      <xs:sequence>
        <xs:element name="capturePoint" type="capturePointType"/>
        <xs:element name="captureArea" type="captureAreaType" 
minOccurs="0"/>
        <xs:any namespace="##other" processContents="lax" minOccurs="0" 
maxOccurs="unbounded"/>
       </xs:sequence>
       <xs:attribute name="sceneIDREF" type="xs:IDREF" use="required"/>
        <xs:anyAttribute namespace="##other" processContents="lax"/>
      </xs:complexType>

* Media capture type still presents a <choice> element allowing to 
distinguish between spatially definible captures and non spatially 
definible captures.

          <xs:choice>
              <xs:sequence>
               <xs:element name="spatialInformation" 
type="tns:spatialInformationType" maxOccurs="unbounded"/>
             </xs:sequence>
              <xs:element name="nonSpatiallyDefinible" type="xs:boolean" 
fixed="true"/>
</xs:choice>

Capture of the last case are for example those related to registrations, 
DVDs, registered presentation, or external streams, that are played in 
the telepresence room and transmitted to remote sites. Spatially 
definible captures are those for which we can provide some spatial 
information in a <spatialInformation> element.

* The <spatialInformation> element is referred to the coordinate space 
of the capture scene indicated in the "sceneIDREF" attribute.
The framework doesn't seem to force a capture to be associated with only 
one scene (or, at least, I don't remember to have read such a constraint 
in the framework document).
If a capture is involved in more than one scene, more than one 
<spatialInformation> should be provided, each one related to the 
coordinate space of the involving scene.

*  We have brought out of the choice section the <composed> element 
because even composed streams can be differentiated in "spatially 
definible" or not.
If we can provide some spatial information for a composed stream, it is 
"spatially definible"; otherwise, we mark that composed stream with a 
<nonSpatiallyDefinible> element.
We can consider the case of the composed PiP video. The framework 
document states that the biggest captured area  should be provided - 
that's a spatial characterization of the capture.
You would have a <composed> capture with a <spatialInformation> element 
with the information of the biggest capture area provided in the 
<spatialInformation>/<captureArea> element. In that case, the 
<capturePoint> would be the vertex of the angular sector corresponding 
to the maximum captured space.
The same rationale can be applied to composed audio captures - you can 
define a <composed> capture with an indication of the captured space in 
the <spatialInformation> element.
You can also choose not to provide any spatial information and describe 
the composed capture as <composed> and <nonSpatiallyDefinible>.
There is also a third alternative, i.e. the one of providing a 
<spatialInformation> for each spatially definible contribution of the 
composed stream, each of such <spatialInformation> elements referring to 
the same capture scene.

What are your feelings about this proposal?

Cheers,

Roberta & Simon

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 26 Oct 2012 15:49:05 -0400

Roberta,

I don't see how this addresses the problem. I think I see why. Comment 
inline

On 10/26/12 6:13 AM, Roberta Presta wrote:
[snip]
> coordinate space of the involving scene.

Whether clear or not, IMO it is *intended* that each capture is 
associated (belongs to) a single scene.

ISTM an obvious way to represent this would be to have an instance of 
mediaCapturesType within captureSceneType, rather than within clueInfoType.

	Thanks,
	Paul

---

From: Roberta Presta <[email-masked]>
Date: Wed, 31 Oct 2012 11:42:36 +0100

Hi Paul,

I don't see any problem in putting media captures inside capture scenes and delete the <mediaCaptures> section under <clueInfo>.
The sceneIDREF attribute of <spatialInformation> does not need to exist anymore, since the coordinate space would be the one of the involving capture scene.
The remaining point now is the new definition of the media capture type, which presents the <choice> section that allows to contain spatial information or,
alternatively, to mark the media capture as "non spatially definible".
What do you think about that proposal?
I copy below the schema snapshot for convenience.

Cheers,

Roberta

      <!-- MEDIA CAPTURE TYPE -->
      <xs:complexType name="mediaCaptureType" abstract="true">
         <xs:sequence>
          <xs:element name="description" type="xs:string" minOccurs="0"/>
          <xs:element name="capturedMedia" type="xs:string" minOccurs="0"/>
          <xs:element name="encGroupIDREF" type="xs:IDREF"/>
          <xs:element name="content" type="xs:string" minOccurs="0"/>
          <xs:element name="switched" type="xs:boolean" minOccurs="0"/>
          <xs:element name="composed" type="xs:boolean" minOccurs="0"/>                                   	
          <xs:element name="maxCaptureEncodings" type="xs:unsignedInt" minOccurs="0"/>
          <xs:choice>
              <xs:sequence>
               <xs:element name="spatialInformation" type="tns:spatialInformationType" maxOccurs="unbounded"/>
             </xs:sequence>             	
          	<xs:element name="nonSpatiallyDefinible" type="xs:boolean" fixed="true"/>         		
          </xs:choice>
          <xs:any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded"/>
         </xs:sequence>
         <xs:attribute name="captureID" type="xs:ID" use="required"/>
         <xs:anyAttribute namespace="##other" processContents="lax"/>
      </xs:complexType>
      
      <!-- SPATIAL INFORMATION TYPE -->
      <xs:complexType name="spatialInformationType">
      <xs:sequence>
        <xs:element name="capturePoint" type="capturePointType"/>
        <xs:element name="captureArea" type="captureAreaType" minOccurs="0"/>
        <xs:any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded"/>
       </xs:sequence>
        <xs:anyAttribute namespace="##other" processContents="lax"/>
      </xs:complexType>

  ----------------------

Roberta,

Whether clear or not, IMO it is *intended* that each capture is 
associated (belongs to) a single scene. ISTM an obvious way to represent 
this would be to have an instance of mediaCapturesType within 
captureSceneType, rather than within clueInfoType.

Thanks,

Paul