Subject: [clue] Clue data model updates

From: Simon Pietro Romano <[email-masked]>
Date: Mon, 15 Oct 2012 17:43:08 +0200

Hi all,

by following the comments we received during the last call, we propose 
to modify the data model as follows:

* New media capture type definition:

        <!-- MEDIA CAPTURE TYPE -->
        <xs:complexType name="mediaCaptureType" abstract="true">
         <xs:sequence>
          <xs:element name="description" type="xs:string" minOccurs="0"/>
          <xs:element name="encGroupIDREF" type="xs:IDREF"/>
          <xs:element name="capturedMedia" type="xs:string"/>
          <xs:element name="content" type="xs:string" minOccurs="0"/>
          <xs:element name="switched" type="xs:boolean" minOccurs="0"/>
          <xs:choice>
              <xs:element name="capturePointIDREF" type="xs:IDREF"/>
              <xs:element name="nonSpatiallyDefinible" type="xs:boolean"/>
              <xs:element name="composed" type="xs:boolean"/>
          </xs:choice>
          <xs:any namespace="##other" processContents="lax"
          minOccurs="0" maxOccurs="unbounded"/>
         </xs:sequence>
         <xs:attribute name="captureID" type="xs:ID" use="required"/>
         <xs:anyAttribute namespace="##other" processContents="lax"/>
        </xs:complexType>

The main change is the insertion of a choice section which is devoted to 
convey the spatial information of the capture:

          <xs:choice>
              <xs:element name="capturePointIDREF" type="xs:IDREF"/>
              <xs:element name="nonSpatiallyDefinible" type="xs:boolean"/>
              <xs:element name="composed" type="xs:boolean"/>
          </xs:choice>

Such a definition requires that only one among the three elements 
(<captureIDREF>, <nonSpatiallyDefinible>, <composed>) can appear inside 
a media capture type.

- case <capturePointIDREF>:
when this element is present inside a media capture, it means that we 
are describing a capture gathered inside the telepresence room from a 
certain capture point. That can be the case of participants' video 
streams, or of the loudest panel stream, etc.
A reference to the capture point must be provided inside the 
<capturePointIDREF> element.
The spatial information represented by the capture point is the minimum 
possible one and it is mandatory for each stream captured in the 
telepresence room that would be sent.
Such a spatial information can be enriched with further information in 
case of video captures: the video capture type, indeed, envisions the 
presence of a <captureArea> element that specifies the area represented 
by the stream.

- case <nonSpatiallyDefinible>:
this is the case for streams associated with registrations, DVDs, 
registered presentation, or external streams, that are played in the 
telepresence room and transmitted to remote sites.
For such streams, it does not exist a capture point inside the room.
This can  also be the case of text streams that are sent from a remote 
site to another one, or for the output analysis document of a 
telemedicine machinary.
Since such streams are not associated with spatial coordinates inside 
the room, it does not make sense provide them with spatial information.
The reproduction of them in a remote site does not need any spatial 
information knowledge.

- case <composed>:
when this element is shown in a media capture, it means that the capture 
is made by mixing more captures together.
This is the case of the telepresence room audio stream composed by the 
participants' audio taken from different camera microphones.
Moreover, this can be the case of a video stream realized by overlapping 
picture-in-picture (PiP) the loudest panel stream with the other 
participants' streams.
Such captures don't have a single point of capture.
As stated in the framework document regarding the PiP video example, the 
biggest area which is represented by such a stream should be indicated. 
The datamodel schema preserves that possibility, since there is a 
<captureArea> element in the video capture type which can be used to 
convey that information.
On the other hand, for mixed audio streams, no spatial information is 
provided.
A possible alternative to that approach could be the one of listing the 
capture points of each component stream. However, this can be onerous 
and not always possible when including in the mix also 
non-spatially-definible streams.

* capture point type and capture axis point
The capture axis point, as envisioned in the framework document, 
provides the direction pointed by the capture device placed in a certain 
capture point.
Since specifying a capture axis point without a capture point does not 
make any sense, the capture point type has a child element named 
<captureAxisPoint>: in such a way a capture axis point can not exist 
without a capture point.
The definition of the capture point type is then the following:

        <!-- CAPTURE POINT TYPE -->
        <xs:complexType name="capturePointType">
         <xs:complexContent>
           <xs:extension base="tns:pointType">
            <xs:sequence>
             <xs:element name="captureAxisPoint" type="tns:pointType"    
minOccurs="0"/>
            </xs:sequence>
            <xs:attribute name="pointID" type="xs:ID"/>
           </xs:extension>
          </xs:complexContent>
          </xs:complexType>

* a new <clueInfo> child: <capturePoints>

In order to list the capture points in a telepresence room, a new child 
element of the clue info type has been added: <capturePoints>.
The capture points therein listed can be referred in the media captures 
through the <captureIDREF> element.
The current aspect of the clue info type is then the following:

       <!-- CLUE INFO TYPE -->
       <xs:complexType name="clueInfoType">
        <xs:sequence>
         <xs:element name="mediaCaptures" type="mediaCapturesType"/>
         <xs:element name="captureScenes" type="captureScenesType"/>
         <xs:element name="encodings" type="encodingsType"/>
         <xs:element name="encodingGroups" type="encodingGroupsType"/>
         <xs:element name="simultaneousSets" type="simultaneousSetsType"/>
         <xs:element name="capturePoints" type="capturePointsType"/>
         <xs:any namespace="##other" processContents="lax" minOccurs="0" 
maxOccurs="unbounded"/>
        </xs:sequence>
        <xs:attribute name="clueInfoID" type="xs:ID" use="required"/>
        <xs:anyAttribute namespace="##other" processContents="lax"/>
       </xs:complexType>

The definition of the capture points type is the following:

       <!-- CAPTURE POINTS TYPE -->
       <xs:complexType name="capturePointsType">
        <xs:sequence>
         <xs:element name="capturePoint" type="capturePointType" 
maxOccurs="unbounded"/>
         <xs:element name="descritption" type="xs:string" minOccurs="0"/>
        </xs:sequence>
        <xs:attribute name="scale" type="scaleType" use="required"/>
       </xs:complexType>

Comments are welcome.

Cheers,

Roberta & Simon

-- 
                             _\\|//_
                             ( O-O )
    ~~~~~~~~~~~~~~~~~~~~~~o00~~(_)~~00o~~~~~~~~~~~~~~~~~~~~~~~~
                     Simon Pietro Romano
               Universita' di Napoli Federico II
                  Computer Science Department
         Phone: +39 081 7683823 -- Fax: +39 081 7684219
                 e-mail: [email-masked]
           http://www.comics.unina.it/simonpietro.romano

     <<Molti mi dicono che lo scoraggiamento è l'alibi degli
    idioti. Ci rifletto un istante; e mi scoraggio>>. Magritte.
                          oooO
    ~~~~~~~~~~~~~~~~~~~~~~(   )~~ Oooo~~~~~~~~~~~~~~~~~~~~~~~~~
                           \ (    (   )
                            \_)    ) /
                                  (_/

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 15 Oct 2012 12:27:52 -0400

(as individual)

Simon & Roberta,

For the most part this seems good. I have a few comments inline.

	Thanks,
	Paul

On 10/15/12 11:43 AM, Simon Pietro Romano wrote:

[snip]

> On the other hand, for mixed audio streams, no spatial information is

Now that you have detailed the video case, ISTM that there ought to be 
*some* spatial information about audio. (Disclaimer: I know *nothing* 
about audio processing.) I realize that audio can't be bounded the way 
video is. But surely there is a big difference between an 
omni-directional mic at the capture point and a mic that is focused 
along the capture axis. Shouldn't that distinction be captured? (I 
realize this is primarily a question about the framework, not the data 
model.)

[snip]

>         <xs:attribute name="scale" type="scaleType" use="required"/>

I think this mishandles some of the semantic constraints in the framework.

An advertisement contains multiple scenes. Each scene has a single 
scaletype and coordinate space. While a capture can be present in 
multiple capture sets, it can only be in the capture sets of a single 
scene. That constraint isn't reflected in the model. If it were, then 
there would be a separate set of capturepoints for each scene, and no 
need to specify the scaletype within capturePointsType. As written above 
all capture points, even from multiple scenes, must have the same scale 
type, and so all scenes are also forced to have the same scale type.