Subject: [clue] Text media in clue

From: Gunnar Hellström <[email-masked]>
Date: Fri, 05 Oct 2012 23:41:37 +0200

I am sorry, I have not followed clue closely since long.
When I now rapidly browsed the framework, I noticed that there is only 
mentioning of the video and audio media. In an early discussion we found 
it feasible to include also the third common real-time medium - text.

All conference-like tools today have some kind of text communication. I 
suggest that also CLUE gets its explicit specification about how text is 
handled.

We had some very early discussions about this topic, but we had at that 
time a bit hard to decide what features of text was clue specific and 
what was more normal multimedia communication specific. With the 
experience and structure you have now, I think it will be easy for you 
to amend the specifications to include at least the real-time text 
medium in a fruitful way.

I made a brief description about how I imagine real-time text might be 
used in a clue environment.
Is this a description suitable to start with with the goal to include 
what is needed for the text medium in clue specifications?

-----Description of use of text in a clue 
environment--------------------------

Text belongs to a modern conference environment. Not only for 
accessibility.
It has special characteristics such as

-positioning in general
-positioning in relation to specific video presentation
-positioning overlaid or not with video, or in unrelated area
-size of text,
-size of area to display in
-background and foreground colour
-visual effects to use if presented overlayed on video
-source address or nickname
-source role
-joint display with other sources or not
-language
-script
-scope of real-time edits
-division in messages
-label style

For many of these parameters, there is interest to have a default value 
assigned by the conference organizer or the source, and a viewer value 
assigned by the viewer. The viewer value should in most cases dominate 
over the organizer value.

Characteristics that should have this double source are:
-positioning in general
-positioning in relation to specific video presentation
-positioning overlaid or not with video, or in unrelated area
-size of text,
-size of area to display in
-background and foreground colour
-visual effects to use if presented overlayed on video
-joint display with other sources or not
-label style

Some simple use cases:

1. Each participant contribute text to a text box underneath their 
video. Used for comments, facts that need exact spelling, cut and paste 
material. Text flows in real-time as typed in order to minimize wait time.

2. Each participant contribute text to a text box underneath their 
video. Used for typed contributions by participants who do not speak or 
sign, or to be understood by participants who do not hear.

3. All participants contribute to text display in a common text box. 
Contributions are labeled. Display may be in real-time so that many 
entries are built up in real-time simultaneously and moved to a history 
area when messages are completed.

4. One single source of text, a text interpreter, listening to mixed 
audio and producing text. Text displayed by default underneath the video 
view of the speaking or signing person.

5. One single source of text, a text interpreter, listening to mixed 
audio from the speaking participants and producing text. Text displayed 
in one statically placed text box. Source indicated in labels mixed with 
the text.

6. One single source of text, a text interpreter, listening to mixed 
audio from the speaking participants and producing text. Text displayed 
in one statically placed text box underneath the video of a sign 
language interpreter. Source indicated in labels mixed with the text.

7. Multiple sources of text in different labeled languages, a text 
interpreter per language, listening to mixed audio from the speaking 
participants and producing text. Text displayed in one statically placed 
text box per language. Language indicated on the text box. Source 
indicated in labels mixed with the text.

Regards
Gunnar

-- 

___________________________________________________
Gunnar Hellström
Omnitor
[email-masked]
+46708204288

---

From: Christian Groves <[email-masked]>
Date: Tue, 09 Oct 2012 11:31:08 +1100

Hello Gunnar,

A few comments/questions please see my responses below marked [CNG].

Regards, Christian

On 6/10/2012 7:41 AM, Gunnar Hellström wrote:
> mentioning of the video and audio media. In an early discussion we 
> found it feasible to include also the third common real-time medium - 
> text.

> All conference-like tools today have some kind of text communication. 
> I suggest that also CLUE gets its explicit specification about how 
> text is handled.

[CNG] Yes the framework currently only has audio and video based 
captures. I would agree that there should be something included in the 
framework to discuss the relationship to text. To what level I think 
needs to be discussed further.
>
> We had some very early discussions about this topic, but we had at 
> that time a bit hard to decide what features of text was clue specific 
> and what was more normal multimedia communication specific. With the 
[CNG] You've probably seen my draft 
<draft-groves-clue-scene-clarifications-00> I think this goes some way 
in supporting multimedia conferencing scenarios where text is used. 
However this is in light of text associated with a video capture, not as 
a separate media stream.

> It has special characteristics such as
[CNG] I think what needs to be considered above is which of the 
characteristics are applicable to a CLUE advertisement and what would be 
applicable to being associated with the media stream. I think CLUE 
information falls largely into two categories: 1. information used to 
determine what captures to receive and 2. information used to render the 
> -positioning in general
[CNG] How do you see this as being related to the current spatial 
attributes in CLUE? Given that a text stream doesn't exist in a physical 
space is this positioning information associated with where the person 
generating the text stream is?

> -source address or nickname
[CNG] It seems to me that these are largely consumer determined 
attributes. They don't seem to be the sort of thing that you would base 
the decision to use one advertised capture over another one on. I guess 
these could be attributes associated with the media stream itself i.e. 
signalled via SDP. Is there any support in SDP for these sorts of things 
today?

> -source role
[CNG] My draft on CLUE attributes has a parameter for role. Is this the 
sort of thing you are talking about?

> -joint display with other sources or not
[CNG] What do you have in mind here? Are you talking about mixing 
multiple text streams or by other sources do you mean audio and video?

> -script
[CNG] My draft on CLUE attributes has a parameter for language. I think 
that this also supports script unless you have something else in mind?

> -scope of real-time edits
[CNG] Do you have more information on this? From initial reading it 
seems like this would be more suitable in SDP.
> -label style
[CNG] Again I'm not sure what these relate to. From initial reading it 
seems something for SDP.

>
> For many of these parameters, there is interest to have a default 
> value assigned by the conference organizer or the source, and a viewer 
> value assigned by the viewer. The viewer value should in most cases 
> dominate over the organizer value.
>
[CNG] I think this could fall into the realm of SDP. i.e. Once the 
support of a text capture is decided these specific attributes are sent 
from the provider to the consumer in the SDP setting up the media stream.
> video. Used for comments, facts that need exact spelling, cut and 
> paste material. Text flows in real-time as typed in order to minimize 
> wait time.
>
> video. Used for typed contributions by participants who do not speak 
> or sign, or to be understood by participants who do not hear.
[CNG] I guess the technical requirement for the above two cases is the 
ability to link a text stream with a particular CLUE video capture. This 
could be done by assigning a spatial region to a text capture or could 
be done at a later stage via SDP.

> entries are built up in real-time simultaneously and moved to a 
> history area when messages are completed.
[CNG] I'm not sure that there's a impact for CLUE in this case.
> audio and producing text. Text displayed by default underneath the 
> video view of the speaking or signing person.
[CNG] I think this could be largely supported by the attributes in my 
draft.
> audio from the speaking participants and producing text. Text 
> displayed in one statically placed text box. Source indicated in 
> labels mixed with the text.
[CNG] I'm not sure there's an impact on CLUE here. There's no spatial 
relation and it doesn't appear this would drive the selection of 
audio/video captures.
> audio from the speaking participants and producing text. Text 
> displayed in one statically placed text box underneath the video of a 
> sign language interpreter. Source indicated in labels mixed with the 
> text.
[CNG] I think the decision of where to display the text i.e. under the 
sign language interpreter could be driven by a language ("sl") capture 
parameter value. It doesn't seem that there's other impacts.
> participants and producing text. Text displayed in one statically 
> placed text box per language. Language indicated on the text box. 
> Source indicated in labels mixed with the text.

[CNG] I guess if a TEXT capture was introduced and multiple languages 
were offered then the language attribute could be used to select the 
language/s to be received.