Subject: [Slim] [slim] #6 (negotiating-human-language): Coolect information on modality in separate section for clarity

From: "slim issue tracker" <[email-masked]>
Date: Mon, 01 Aug 2016 22:23:55 -0000

#6: Coolect information on modality in separate section for clarity

 Introduce a new section on implicit and explicit modality and move
 modaliy-related information to this section

 Change 1
 _________________________________________________________________________________________________________________

 6.x Implicit and explicit modality

 The use of the "humintlang-send" and "humintlang-recv" media attributes is
 supposed to indicate combinations of media stream, modality, favoured
 level language(s) and preference for each language within the media
 stream.
 Spoken and written languages have common language tags. Sign languages
 have unique language tags. Thus, in many cases, the language tag itself
 does not indicate modality explicitly.

 Some media types indicate implicitly the dominating modality used in
 streams of that media type. Thus, for 'audio' media, the default modality
 is 'spoken'. For 'video' media, the default modality is 'signed' when
 using sign language tags and 'spoken' in the form of a view of the
 speaking person when using spoken/written language tags. For 'text' media,
 the default modality is 'written'.
 Other media types, such as 'application' have less obvious relation to any
 specific modality.
 Therefore, a "modality" parameter is available in the media attributes for
 explicit indication of the modality of language use in the indicated
 media.

 See the syntax section for syntax description for the modality parameter.

 The following table indicates the modalities related to the foreseen
 combinations of media type and language tag types.
 -----------------------------------------------------------------------------------------------------------------------
 |Media type    subtype and subprotocol   language tag type
 modality parameter   modality
 |----------------------------------------------------------------------------------------------------------------------
 |Audio         -                         spoken or written         none or
 spoken       spoken
 |Video         -                         signed                    none or
 signed       signed
 |Video             -                         spoken or written
 none or spoken       spoken (view of speaking person)
 |Text          -                         spoken or written         none or
 written      written
 |Application   webrtc-datachannel(t140)  spoken or written         written
 written
 -----------------------------------------------------------------------------------------------------------------------

 Note: The application/webrtc-datachannel subprotocol 't140' is specified
 in draft-schwarz-mmusic-t140-usage-data-channel.
 -------------------------------------------------------------------------------------------------------

 -------------------------------------------------------------------------------------
 Change 2:  Move the following paragraph from section 6.2 to this new
 section 6.x:
 ------------------------------------------------------------------------------
 "   Note that while signed language tags are used with a video stream to
    indicate sign language, a spoken language tag for a video stream in
    parallel with an audio stream with the same spoken language tag
    indicates a request for a supplemental video stream to see the
    speaker."
 ------------------------------------------------------------------------------

 Change 3: Delete the following statement from section 6.4. It is the same
 as above but less clearly stated.
 ----------------------------------------------------------------------
    A spoken language tag for a video stream in conjunction with an audio
    stream with the same language might indicate a request for
    supplemental video to see the speaker.
 ----------------------------------------------------------------------

 For discussion on this issue: If text embedded as captions in video is a
 modality that should be covered, then one line with explicit modality =
 "written" needs to be added to the table above.

-- 
-------------------------------------+-------------------------------------
 Reporter:                           |      Owner:  draft-ietf-slim-
  [email-masked]        |  [email-masked]
     Type:  enhancement              |     Status:  new
 Priority:  major                    |  Milestone:
Component:  negotiating-human-       |    Version:
  language                           |   Keywords:  modality
 Severity:  In WG Last Call          |
-------------------------------------+-------------------------------------

Ticket URL: <https://trac.tools.ietf.org/wg/slim/trac/ticket/6>
slim <https://tools.ietf.org/slim/>

---

From: "slim issue tracker" <[email-masked]>
Date: Mon, 01 Aug 2016 22:24:43 -0000

#6: Collect information on modality in separate section for clarity

  language                           |  Resolution:
 Keywords:  modality                 |

Ticket URL: <https://trac.tools.ietf.org/wg/slim/trac/ticket/6#comment:1>
slim <https://tools.ietf.org/slim/>

---

From: "slim issue tracker" <[email-masked]>
Date: Mon, 01 Aug 2016 22:31:23 -0000

Comment (by [email-masked]):

 Sorry, the table got unreadable. I will need to edit it.

Ticket URL: <https://trac.tools.ietf.org/wg/slim/trac/ticket/6#comment:2>
slim <https://tools.ietf.org/slim/>

---

From: "slim issue tracker" <[email-masked]>
Date: Tue, 02 Aug 2016 07:32:14 -0000

New edit of the table:

 ||Media type||subtype and subprotocol||language tag type||modality
 parameter||modality||
 ||Audio||-||spoken or written||none or spoken||spoken||
 ||Video||-||signed||none or signed||signed||
 ||Video||-||spoken or written||none or spoken||spoken (view of speaking
 person)||
 ||Text||-||spoken or written||none or written||written||
 ||Application||webrtc-datachannel(t140)||spoken or
 written||written||written||

Ticket URL: <https://trac.tools.ietf.org/wg/slim/trac/ticket/6#comment:3>
slim <https://tools.ietf.org/slim/>

---

From: "slim issue tracker" <[email-masked]>
Date: Mon, 15 Aug 2016 16:50:33 -0000

Comment (by [email-masked]):

 I'm a bit confused by the meaning of "modality parameter".  Are you
 proposing that an assumption be made about language if signing is not
 explicitly indicated? For example, if there is no language indicated for a
 video m-line, or just "English" (but not ASL), what should we assume "by
 default"?

Ticket URL: <https://trac.tools.ietf.org/wg/slim/trac/ticket/6#comment:4>
slim <https://tools.ietf.org/slim/>

---

From: "slim issue tracker" <[email-masked]>
Date: Mon, 15 Aug 2016 17:00:41 -0000

Comment (by [email-masked]):

 Chairs: this is a summary of the other issues, so is invalid, will close.

Ticket URL: <https://trac.tools.ietf.org/wg/slim/trac/ticket/6#comment:5>
slim <https://tools.ietf.org/slim/>

---

From: "slim issue tracker" <[email-masked]>
Date: Mon, 15 Aug 2016 17:00:53 -0000

Changes (by [email-masked]):

 * status:  new => closed
 * resolution:   => invalid

-- 
-------------------------------------+-------------------------------------
 Reporter:                           |       Owner:  draft-ietf-slim-
  [email-masked]        |  [email-masked]
     Type:  enhancement              |      Status:  closed
 Priority:  major                    |   Milestone:
Component:  negotiating-human-       |     Version:
  language                           |  Resolution:  invalid
 Severity:  In WG Last Call          |
 Keywords:  modality                 |
-------------------------------------+-------------------------------------

Ticket URL: <https://trac.tools.ietf.org/wg/slim/trac/ticket/6#comment:6>
slim <https://tools.ietf.org/slim/>

---

From: Gunnar Hellström <[email-masked]>
Date: Tue, 16 Aug 2016 00:42:36 +0200

It must be a mistake to close #6 saying that it is a summary. It is #10 
that is a summary and can be closed.

Please revive #6. It collects information on modality that was spread in 
various sections and was treated quite informally. It also adds information.

/Gunnar

-- 
-----------------------------------------
Gunnar Hellström
Omnitor
[email-masked]
+46 708 204 288

---

From: Gunnar Hellström <[email-masked]>
Date: Tue, 16 Aug 2016 01:03:28 +0200

>   video m-line, or just "English" (but not ASL), what should we assume "by
The most important need for the modality parameter is to tell in media 
sections for a media that has no explicit modality, ( like m=application 
) in what modality the language is supposed to be used.

In WebRTC, we will have real-time text multiplexed in a subchannel of a 
data channel. There is no m=text there, so we need to apply a parameter 
saying that the language is used in written modality.

So, normally the modality parameter goes together with a list of 
languages intended to be used in that modality in that media in a 
specified direction.

What you ask about is probably a side effect I have thought about for a 
long time and introduced here. It is that it in many cases can be 
sufficient to indicate that a specific modality is preferred, and the 
language is given by being default for the service or the country. That 
is why all languages are optional in the language list in the proposed 
syntax. This side effect may be seen as a complication and could be 
deleted, saying that you must always indicate at least one language in a 
humintlang attribute.

There are joint text and sign language relay services which currently 
start with asking their users if they want to use text or sign language 
( they serve total conversation users ). But they might only serve one 
language area with one written language and one sign language. This 
extra question could be eliminated if the terminal indicated the 
intended modality, and no language because it is a default for that service.

/Gunnar

---

From: "slim issue tracker" <[email-masked]>
Date: Tue, 16 Aug 2016 21:40:03 -0000

The reason to close this one was misplaced.

 It is issue #10 that is a summary. This one is a unique effort to collect
 information about modality in one place because at the moment it is
 scattered and unclear and repeated.

 Please reopen.

Ticket URL: <https://trac.tools.ietf.org/wg/slim/trac/ticket/6#comment:7>
slim <https://tools.ietf.org/slim/>

---

From: Brian Rosen <[email-masked]>
Date: Tue, 16 Aug 2016 17:53:55 -0400

I think it should remain closed.  I don’t think this reorganization would be particularly helpful if we don’t implement all you modality extensions (per chair direction).

I think it would be a really bad idea to try to label applications by modality.  Just allow our tags on the m=application SDP.  That’s all we need.

Brian

> On Aug 16, 2016, at 5:40 PM, slim issue tracker <[email-masked]> wrote:
> 
> _______________________________________________
> SLIM mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/slim

---

From: "slim issue tracker" <[email-masked]>
Date: Wed, 17 Aug 2016 06:56:45 -0000

If the new section is not accepted, we need to make at least three changes
 in the draft corresponding to change proposal 1, 2, and 3 above.

 1. Clearly specify that the humintlang attributes are allowed in the sdp
 specification of the application/webrtc-datachannel t140 subchannel and
 that humintlang attributes in that subchannel specification shall be
 understood as indication of written modality.
 The humintlang attributes will for that case be placed in sdp "a=dcsa:"
 attributes for that subchannel. That will also need to be stated in the
 formal rfc4566bis -formatted specification of the humintlang attributes.
 Since we know this usage for the written modality, the usage level dcsa
 must also be registered by IANA for the humintlang attributes, ( see 8.3
 in draft-ietf-mmusic-data-channel-sdpneg )

 Text proposal for 6.2 if not any new 6.x is created:

 "The t140 subchannel of a media stream specified by MIME type/subtype
 "application/webrtc-datachannel" is intended to be used for written
 modality. Capability to use this subchannel for human language
 communication shall be specified by placing the corresponding humintlang-
 send and humintlang-recv attributes in a=dcsa: attributes in the sdp
 specification of the subchannel.

 2. Modify the following paragraph in 6.2:

 to
 " While signed language tags are used with a video stream to
 indicate sign language, a spoken/written language tag for a video stream
 indicates a view of a speaking person."

 Motivations for this change: a) The leading words "Note that" may be
 understood as start of less important informational information. b)It is
 not only in requests that this notation can be used, therefore the words
 "a request" are deleted. c) The requirement to have the same language in
 the audio stream is a complication that the users can take care of without
 the need to complicate the mechanism, therefore that requirement is
 deleted.

 However, there is another possible use of spoken/written language tags for
 video media. That is for text captions overlaid in the video stream. That
 is often used for stored and replayed media, but less often for
 conversational use. Are we sure we want to ignore that way of transmitting
 text? If not, a "written" modality parameter is needed for this case.

 3. Section 6.4 Silly states has this paragraph:

 "A spoken language tag for a video stream in conjunction with an audio
 stream with the same language might indicate a request for
 supplemental video to see the speaker."

 This is the same information as discussed under 2 above, but placed under
 the header "Silly states". It is not a silly state and it is fully
 described in 6.2, so I suggest to delete it.

Ticket URL: <https://trac.tools.ietf.org/wg/slim/trac/ticket/6#comment:8>
slim <https://tools.ietf.org/slim/>

---

From: Gunnar Hellström <[email-masked]>
Date: Wed, 17 Aug 2016 09:17:26 +0200

> I think it should remain closed.  I don’t think this reorganization would be particularly helpful if we don’t implement all you modality extensions (per chair direction).
If the new section is not accepted, we need to make at least three 
changes in the draft corresponding to change proposal 1, 2, and 3 in 
issue #6.
I just posted proposals for these changes as comments to #6. I hope they 
are considered even if the state of the issue is currently closed. I 
still want it to be opened again.

The intention of my change proposals and introduction of the modality 
section is to bring clarity about use of modality. The current draft 
(-04) has some weaknesses in that area. It has nothing to do with the 
need to also put priority on modalities that is dealt with in issue #8.

> I think it would be a really bad idea to try to label applications by modality.  Just allow our tags on the m=application SDP.  That’s all we need.
So you mean that it is sufficient to register by IANA that 
humintlang-send and huminlang-recv are allowed in general in a=dcsa:  
attributes for placement in WebRTC subprotocols and the modality should 
be clear from what subprotocol it is used with.   Or the modality may 
need to be specified in the subprotocol spec?
The only one I know currently that we have this need for is the t140 
subprotocol. But others may appear. I thought the modality parameter in 
the attribute value was useful for indication of what modality is meant 
in either ambigous cases as video, and in less known cases as some 
m=application cases.

There is another modality that we have not discussed at all. That is the 
"text message" modality. It is not really interactive, but very often 
used together with interactive media in a kind of slow 
pseudo-conversational mode. One implmentation is MSRP that is specified 
by the m=message media. Is there any interest to extend our 
specification to handle that modality as well?

/Gunnar

---

From: Brian Rosen <[email-masked]>
Date: Wed, 17 Aug 2016 09:25:46 -0400

> On Aug 17, 2016, at 3:17 AM, Gunnar Hellström <[email-masked]> wrote:
> 
> If the new section is not accepted, we need to make at least three changes in the draft corresponding to change proposal 1, 2, and 3 in issue #6.
> I just posted proposals for these changes as comments to #6. I hope they are considered even if the state of the issue is currently closed. I still want it to be opened again.
> 
> The intention of my change proposals and introduction of the modality section is to bring clarity about use of modality. The current draft (-04) has some weaknesses in that area. It has nothing to do with the need to also put priority on modalities that is dealt with in issue #8.
> 
> 
> 
> 
>> 
> So you mean that it is sufficient to register by IANA that humintlang-send and huminlang-recv are allowed in general in a=dcsa:  attributes for placement in WebRTC subprotocols and the modality should be clear from what subprotocol it is used with.   Or the modality may need to be specified in the subprotocol spec?
> The only one I know currently that we have this need for is the t140 subprotocol. But others may appear. I thought the modality parameter in the attribute value was useful for indication of what modality is meant in either ambigous cases as video, and in less known cases as some m=application cases.
I’m not sure we need to specify the a= line the tags go on.  I think it’s just sufficient to say we can put a humintlang tag on an m=application SDP section.  I think the modality will be clear in context.
> 
> 
> There is another modality that we have not discussed at all. That is the "text message" modality. It is not really interactive, but very often used together with interactive media in a kind of slow pseudo-conversational mode. One implmentation is MSRP that is specified by the m=message media. Is there any interest to extend our specification to handle that modality as well?
Yeah, I’d like to do that.

> [email-masked] <mailto:[email-masked]>
> +46 708 204 288