Subject: [Rum] Real-time text in WebRTC is discussed in mmusic - a topic closely telated to rum

From: Gunnar Hellström <[email-masked]>
Date: Tue, 27 Aug 2019 15:52:23 +0200

Hi,

A topic is currently discussed in mmusic that is closely related to rum. 
it is WebRTC transport of real-time text.

The draft is draft-holmberg-mmusic-t140-usage-data-channel .

A good point to start reading could be:

https://mailarchive.ietf.org/arch/browse/mmusic/?gbt=1&q=draft-holmberg-mmusic-t140-usage-data-channel

Please check if the current state of the discussion suits rum!

The only issue that seems to be remaining is how to transport RTT data 
to and from a conference server that combines all traffic per media in a 
meeting in one data stream. That is not very elegantly specified for RFC 
4103 transport of RTT in RTP either, so we might want to do a rapid 
action together to solve the multi-party RTT MCU case in a general and 
consistent way.

Regards

Gunnar

-- 
-----------------------------------------
Gunnar Hellström
Omnitor
[email-masked]
+46 708 204 288

---

From: Brian Rosen <[email-masked]>
Date: Tue, 27 Aug 2019 15:48:59 -0400

The problem of conference 4103 RTT is high on my list of work I need to get done.  So, I’m motivated to help out.
The basic problem is that we’re going to get very inconsistent UI doing it that way, because of how systems will handle backspace of one party that extends beyond responses from other parties:

Alice: I waited for you
Bob: I didn’t see you
Alice: sorry

And then Alice types 12 backspaces.

What should happen?

Brian

> On Aug 27, 2019, at 9:52 AM, Gunnar Hellström <[email-masked]> wrote:
> 
> Hi,
> 
> A topic is currently discussed in mmusic that is closely related to rum. it is WebRTC transport of real-time text.
> 
> The only issue that seems to be remaining is how to transport RTT data to and from a conference server that combines all traffic per media in a meeting in one data stream. That is not very elegantly specified for RFC 4103 transport of RTT in RTP either, so we might want to do a rapid action together to solve the multi-party RTT MCU case in a general and consistent way.
> 
> Regards
> 
> Gunnar
> 
> --

---

From: Gunnar Hellström <[email-masked]>
Date: Tue, 27 Aug 2019 22:43:13 +0200

> The problem of conference 4103 RTT is high on my list of work I need to get done.  So, I’m motivated to help out.
Thanks, great.
> The basic problem is that we’re going to get very inconsistent UI doing it that way, because of how systems will handle backspace of one party that extends beyond responses from other parties:
(well, for me the currently most basic problem is to have a reliable way 
to append received text to the already presented text of the right 
participant. And that is getting worse in WebRTC than it was in RFC 
4103. But we will sort it out.)
> And then Alice types 12 backspaces.

You are right that there are a number of ways to handle the RTT UI. And 
just as inconsistencies are common with a message oriented UI, where 
messages show up in a confusing order because two users completed 
messages in an unexpected time order, it is possible that RTT text gets 
displayed in a strange order after erasure and retyping. It is better 
for RTT than for message oriented presentation, and user get used to it 
in both cases.  With the labelled style in one column you have in the 
example, I would recommend that first 5 backspaces erase "sorry", next 
backspace erases the line separator, and pulls down "I waited for you" 
to be shown last, as an uncompleted text. Then the next 6 backspaces 
erase so that only "I waited f" is displayed. When Alice adds text and 
end with a new line, the corrected sentence is allowed to flow up when 
new text is added from any participant.  That causes a bit strange 
order, but it is just as manageable as when text in messaging 
applications appear in an unexpected order so that one message seems to 
be a respone on something totally else than what was intended.

A sophisticated UI may mark text that is moved and modified.

We want to keep sentences or at least phrases from each participant 
together in a readable unit. Already that causes a design decision on 
where to place the completed chunk of text once the user has completed 
it. The start of the chunk may be older than completed text from other 
participants which would motivate to move it up a bit in the 
presentation. But the end of it is at that moment the latest text to 
present. I think it is best to let the finished text be presented last 
on the display, but let others' newer text push everything up and be 
displayed last.

T.140 has information on how to handle erasure:

-------------------From T.140---------------------------

8.2 Erase last character
Purpose: Erase the last character sent from the display at the receiving 
end.
Code: BS: 0008.
Procedure: On the receiving end: Move the insertion point to the last 
character and erase it.
Combined characters are erased as a unit, with one BS erasing the whole 
character even if it is
combined from more than one component.
Control sequences (like CR LF) are erased in one operation.
NOTE – The same action shall be taken on the local display.

------------------------------------------------------------

   /Gunnar

---

From: Brian Rosen <[email-masked]>
Date: Tue, 27 Aug 2019 17:15:58 -0400

At least by centralizing the problem at a “mixer”, Alice and Bob will see the same thing.

You don’t have the problem in Instant Messaging, because you can’t backspace or delete a sent message.  Of course if multiple people are typing simultaneously in such systems, message order will be confusing in that instant. 

Anyway, we need to specify the mixer for RTT so it receives each of the RTT streams and produces a single composite stream for each participant.

> On Aug 27, 2019, at 4:43 PM, Gunnar Hellström <[email-masked]> wrote:
> 
> (well, for me the currently most basic problem is to have a reliable way to append received text to the already presented text of the right participant. And that is getting worse in WebRTC than it was in RFC 4103. But we will sort it out.)
>> 
>> Alice: I waited for you
>> Bob: I didn’t see you
>> Alice: sorry
>> 
> You are right that there are a number of ways to handle the RTT UI. And just as inconsistencies are common with a message oriented UI, where messages show up in a confusing order because two users completed messages in an unexpected time order, it is possible that RTT text gets displayed in a strange order after erasure and retyping. It is better for RTT than for message oriented presentation, and user get used to it in both cases.  With the labelled style in one column you have in the example, I would recommend that first 5 backspaces erase "sorry", next backspace erases the line separator, and pulls down "I waited for you" to be shown last, as an uncompleted text. Then the next 6 backspaces erase so that only "I waited f" is displayed. When Alice adds text and end with a new line, the corrected sentence is allowed to flow up when new text is added from any participant.  That causes a bit strange order, but it is just as manageable as when text in messaging applications appear in an unexpected order so that one message seems to be a respone on something totally else than what was intended.
> 
> We want to keep sentences or at least phrases from each participant together in a readable unit. Already that causes a design decision on where to place the completed chunk of text once the user has completed it. The start of the chunk may be older than completed text from other participants which would motivate to move it up a bit in the presentation. But the end of it is at that moment the latest text to present. I think it is best to let the finished text be presented last on the display, but let others' newer text push everything up and be displayed last.
> 
> 
> Purpose: Erase the last character sent from the display at the receiving end.
> Code: BS: 0008.
> Procedure: On the receiving end: Move the insertion point to the last character and erase it.
> Combined characters are erased as a unit, with one BS erasing the whole character even if it is
> [email-masked] <mailto:[email-masked]>
> +46 708 204 288

---

From: Gunnar Hellström <[email-masked]>
Date: Wed, 28 Aug 2019 08:15:50 +0200

Den 2019-08-27 kl. 23:15, skrev Brian Rosen:
> You don’t have the problem in Instant Messaging, because you can’t 
> backspace or delete a sent message.  Of course if multiple people are 
> typing simultaneously in such systems, message order will be confusing 
> in that instant.
Right, it is a similar kind of problem that text appears in an 
unexpected order. There is also at least one instant messaging service 
that allows modification in already sent message. But I think it has 
limitations to only accept that in the last message sent. it is 
convenient anyway.
>
> Anyway, we need to specify the mixer for RTT so it receives each of 
> the RTT streams and produces a single composite stream for each 
> participant.

Yes, right, and there is an effort in that direction in:

http://www.realtimetext.org/sites/default/files/Files_and_Documents/Specifications/multiparty-real-time-text-mixer-2011-04-30.pdf

It is written for conference-unaware user devices.

The goals are specified as follows:

The procedures are intended to make best efforts to present a 
multi-party text conversation on a terminal that has no awareness of 
multi-party calls. There are some obvious drawbacks, and a terminal 
designed with multi-party awareness will be able to present multi-party 
call contents in a more flexible way. Only two parties at a time will be 
allowed to display added text in real-time, while the other parties’ 
produced text will need to be stored in the multi-party server for a 
moment awaiting a suitable occasion to be displayed. There are also some 
cases of erasure that will not be performed on the target text but only 
indicated in another way. Even with these drawbacks, the procedure 
provides an opportunity to display text from more than two parties in a 
smooth and readable way.

---------------------------------------------------------------------------------------------------

I see such mixer procedures as a fall-back for cases without conference 
awareness, but want to see support for conference-aware terminals, where 
text from more than two parties can be presented in real-time, and the 
end user or app can have influence over the presentation style - e.g. 
select between the multiple column view and the one-column-with-labels 
view.  A mixer for that case would only need to assure that the receiver 
has the right kind of multi-party awareness and send RTT text with 
source information attached, and let the receiving terminal sort out the 
presentation. This is already possible with CSRC and CNAME when using 
RTP, but we lose that possibility natively when using the WebRTC data 
channel to transport RTT, and would need to specify a way to include the 
source also for that case.

------------------------------------------------------

By the way, what is your current view of how to transport RTT for RUM, 
now when you say that you will use WebRTC transports for media?

Regards

Gunnar

>
>> On Aug 27, 2019, at 4:43 PM, Gunnar Hellström 
>> <[email-masked] <mailto:[email-masked]>> wrote:
>>
>>> The problem of conference 4103 RTT is high on my list of work I need 
>>> to get done.  So, I’m motivated to help out.
>> Thanks, great.
>>> The basic problem is that we’re going to get very inconsistent UI 
>>> doing it that way, because of how systems will handle backspace of 
>>> one party that extends beyond responses from other parties:
>> (well, for me the currently most basic problem is to have a reliable 
>> way to append received text to the already presented text of the 
>> right participant. And that is getting worse in WebRTC than it was in 
>> RFC 4103. But we will sort it out.)
>>>
>>> Alice: I waited for you
>>> Bob: I didn’t see you
>>> Alice: sorry
>>>
>> You are right that there are a number of ways to handle the RTT UI. 
>> And just as inconsistencies are common with a message oriented UI, 
>> where messages show up in a confusing order because two users 
>> completed messages in an unexpected time order, it is possible that 
>> RTT text gets displayed in a strange order after erasure and 
>> retyping. It is better for RTT than for message oriented 
>> presentation, and user get used to it in both cases.  With the 
>> labelled style in one column you have in the example, I would 
>> recommend that first 5 backspaces erase "sorry", next backspace 
>> erases the line separator, and pulls down "I waited for you" to be 
>> shown last, as an uncompleted text. Then the next 6 backspaces erase 
>> so that only "I waited f" is displayed. When Alice adds text and end 
>> with a new line, the corrected sentence is allowed to flow up when 
>> applications appear in an unexpected order so that one message seems 
>> to be a respone on something totally else than what was intended.
>>
>> where to place the completed chunk of text once the user has 
>> completed it. The start of the chunk may be older than completed text 
>> from other participants which would motivate to move it up a bit in 
>> the presentation. But the end of it is at that moment the latest text 
>> to present. I think it is best to let the finished text be presented 
>> last on the display, but let others' newer text push everything up 
>> and be displayed last.
>>
>>
>>>> On Aug 27, 2019, at 9:52 AM, Gunnar Hellström 
>>>> <[email-masked] <mailto:[email-masked]>> 
>>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> A topic is currently discussed in mmusic that is closely related to 
>>>> rum. it is WebRTC transport of real-time text.
>>>>
>>>> The only issue that seems to be remaining is how to transport RTT 
>>>> data to and from a conference server that combines all traffic per 
>>>> media in a meeting in one data stream. That is not very elegantly 
>>>> specified for RFC 4103 transport of RTT in RTP either, so we might 
>>>> want to do a rapid action together to solve the multi-party RTT MCU 
>>>> case in a general and consistent way.
>>>>
>>>> Regards
>>>>
>>>> Gunnar
>>>>
>>>> --

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 29 Aug 2019 12:05:10 -0400

I question how well RTT text is suited to multiparty conferences.

If you have messages on your screen from multiple parties, and many of 
them are updating in real-time, are you going to be able to perceive 
what is going on?

And while you can have a column per person for two-party and maybe 
3-party conversations, that doesn't scale up. With many parties, some 
typing may scroll off the screen before it is complete.

Perhaps for conferences it is better to just use line-at-a-time chat. If 
necessary, I presume there could be gateways between RTT and chat. A RUE 
could have the capability to negotiate down from RTT to chat.

	Thanks,
	Paul

---

From: "Malloy, Jim" <[email-masked]>
Date: Thu, 29 Aug 2019 16:20:24 +0000

Do we intend for the RUE to accommodate multiple party calls (more than three)? I don't recall any language related to anything but a point to point call.  For emergency calls, conferencing in the PSAP (three parties) may be appropriate.  Are we looking for more than that?

--Jim Malloy

-----Original Message-----
From: Rum <[email-masked]> On Behalf Of Paul Kyzivat
Sent: Thursday, August 29, 2019 12:05 PM
To: [email-masked]
Subject: [EXT] Re: [Rum] Real-time text in WebRTC is discussed in mmusic - a topic closely telated to rum

If you have messages on your screen from multiple parties, and many of them are updating in real-time, are you going to be able to perceive what is going on?

And while you can have a column per person for two-party and maybe 3-party conversations, that doesn't scale up. With many parties, some typing may scroll off the screen before it is complete.

Perhaps for conferences it is better to just use line-at-a-time chat. If necessary, I presume there could be gateways between RTT and chat. A RUE could have the capability to negotiate down from RTT to chat.

	Thanks,
	Paul

On 8/28/19 2:15 AM, Gunnar Hellström wrote:
>> typing simultaneously in such systems, message order will be 
>> confusing in that instant.
> http://www.realtimetext.org/sites/default/files/Files_and_Documents/Sp
> ecifications/multiparty-real-time-text-mixer-2011-04-30.pdf
> 
> designed with multi-party awareness will be able to present 
> multi-party call contents in a more flexible way. Only two parties at 
> a time will be allowed to display added text in real-time, while the other parties’
> moment awaiting a suitable occasion to be displayed. There are also 
> some cases of erasure that will not be performed on the target text 
> but only indicated in another way. Even with these drawbacks, the 
> procedure provides an opportunity to display text from more than two 
> parties in a smooth and readable way.
> 
> I see such mixer procedures as a fall-back for cases without 
> conference awareness, but want to see support for conference-aware 
> terminals, where text from more than two parties can be presented in 
> real-time, and the end user or app can have influence over the presentation style - e.g.
> view.  A mixer for that case would only need to assure that the 
> receiver has the right kind of multi-party awareness and send RTT text 
> with source information attached, and let the receiving terminal sort 
> out the presentation. This is already possible with CSRC and CNAME 
> when using RTP, but we lose that possibility natively when using the 
> WebRTC data channel to transport RTT, and would need to specify a way 
> to include the source also for that case.
> 
>>>> The problem of conference 4103 RTT is high on my list of work I 
>>>> need to get done.  So, I’m motivated to help out.
>>> Thanks, great.
>>> right participant. And that is getting worse in WebRTC than it was 
>>> in RFC 4103. But we will sort it out.)
>>>>
>>>> Alice: I waited for you
>>>> Bob: I didn’t see you
>>>> Alice: sorry
>>>>
>>> together in a readable unit. Already that causes a design decision 
>>> on where to place the completed chunk of text once the user has 
>>> completed it. The start of the chunk may be older than completed 
>>> text from other participants which would motivate to move it up a 
>>> bit in the presentation. But the end of it is at that moment the 
>>> latest text to present. I think it is best to let the finished text 
>>> be presented last on the display, but let others' newer text push 
>>> everything up and be displayed last.
>>>
>>>
>>> Combined characters are erased as a unit, with one BS erasing the 
>>> whole character even if it is combined from more than one component.
>>>>> A topic is currently discussed in mmusic that is closely related 
>>>>> to rum. it is WebRTC transport of real-time text.
>>>>>
>>>>> want to do a rapid action together to solve the multi-party RTT 
>>>>> MCU case in a general and consistent way.
>>>>>
>>>>> Regards
>>>>>
>>>>> Gunnar
>>>>>
>>>>> --

--
Rum mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/rum

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 29 Aug 2019 12:57:58 -0400

Hi Jim,

> Do we intend for the RUE to accommodate multiple party calls (more than three)? I don't recall any language related to anything but a point to point call.  For emergency calls, conferencing in the…

Good question.

Speaking as an individual, not chair:

IIUC, VRS service (with a CA) explicitly excludes conference calls. But 
I don't think that extends to P2P video calls. Though there is no 
mechanism provided for establishing a conference, it should be 
acceptable using mechanisms outside of the VRS infrastructure.

Since a P2P call can only be established between two parties that have 
entries in iTRS, it would seem to require something that has qualified 
to get a number in there, that can properly interface. Perhaps something 
that totally meets the specs for a RUE.

So it is all hypothetical at the moment. How deep we want to get into 
this hypothetical case remains to be determined. I'm inclined to think 
it should be out of scope for now.

What do others think?

	Thanks,
	Paul

---

From: Brian Rosen <[email-masked]>
Date: Thu, 29 Aug 2019 13:14:52 -0400

“Line at a time” could be an implementation option.  The protocol mechanism is all streams head to a mixer and the mixer sends a single stream to each participant.

Gunnar, we agreed we were using 4103 in RUM.

Brian

> On Aug 29, 2019, at 12:05 PM, Paul Kyzivat <[email-masked]> wrote:
> 
>> Right, it is a similar kind of problem that text appears in an unexpected order. There is also at least one instant messaging service that allows modification in already sent message. But I think it has limitations to only accept that in the last message sent. it is convenient anyway.
>>> 
>> The procedures are intended to make best efforts to present a multi-party text conversation on a terminal that has no awareness of multi-party calls. There are some obvious drawbacks, and a terminal designed with multi-party awareness will be able to present multi-party call contents in a more flexible way. Only two parties at a time will be allowed to display added text in real-time, while the other parties’ produced text will need to be stored in the multi-party server for a moment awaiting a suitable occasion to be displayed. There are also some cases of erasure that will not be performed on the target text but only indicated in another way. Even with these drawbacks, the procedure provides an opportunity to display text from more than two parties in a smooth and readable way.
>> I see such mixer procedures as a fall-back for cases without conference awareness, but want to see support for conference-aware terminals, where text from more than two parties can be presented in real-time, and the end user or app can have influence over the presentation style - e.g. select between the multiple column view and the one-column-with-labels view.  A mixer for that case would only need to assure that the receiver has the right kind of multi-party awareness and send RTT text with source information attached, and let the receiving terminal sort out the presentation. This is already possible with CSRC and CNAME when using RTP, but we lose that possibility natively when using the WebRTC data channel to transport RTT, and would need to specify a way to include the source also for that case.
>> By the way, what is your current view of how to transport RTT for RUM, now when you say that you will use WebRTC transports for media?
>> Regards
>> Gunnar
>>> 
>>>> On Aug 27, 2019, at 4:43 PM, Gunnar Hellström <[email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>>>> 
>>>>>> On Aug 27, 2019, at 9:52 AM, Gunnar Hellström <[email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> https://mailarchive.ietf.org/arch/browse/mmusic/?gbt=1&q=draft-holmberg-mmusic-t140-usage-data-channel <https://mailarchive.ietf.org/arch/browse/mmusic/?gbt=1&q=draft-holmberg-mmusic-t140-usage-data-channel>
>>>>>> 
>>>> [email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>
>>>> +46 708 204 288
>>> 
>>> 
>> -- 
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/rum <https://www.ietf.org/mailman/listinfo/rum>

---

From: Brian Rosen <[email-masked]>
Date: Thu, 29 Aug 2019 13:17:45 -0400

I think we want a rum device to be able to be a good conference participant, but like nearly all conference participants, they only see a 2 party call and the bridge does the magic.
I don’t think we need to require anything else.  I think we want a bridge to be able to host a rum-compliant endpoint.

Brian

> On Aug 29, 2019, at 12:20 PM, Malloy, Jim <[email-masked]> wrote:
> 
> From: Rum <[email-masked] <mailto:[email-masked]>> On Behalf Of Paul Kyzivat
> To: [email-masked] <mailto:[email-masked]>
>>>> <[email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>>>> 
>>>>>> <[email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>>
>>>>>> wrote:
>>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> https://mailarchive.ietf.org/arch/browse/mmusic/?gbt=1&q=draft-hol <https://mailarchive.ietf.org/arch/browse/mmusic/?gbt=1&q=draft-hol>
>>>>>> mberg-mmusic-t140-usage-data-channel
>>>>>>

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 29 Aug 2019 14:06:58 -0400

On 8/29/19 1:17 PM, Brian Rosen wrote:
> I think we want a rum device to be able to be a good conference 
> participant, but like nearly all conference participants, they only see 
> a 2 party call and the bridge does the magic.
> I don’t think we need to require anything else.  I think we want a 
> bridge to be able to host a rum-compliant endpoint.

I agree conceptually. But we need to recognize that we are out ahead of 
VRS. Near term there is no way to get an audio/video/RTT bridge into a 
call with a VRS user. Because the bridge is not in the VRS iTRS it will 
only be able to get connected as an audio call with and interpreter.

The first form of conference is likely to be for emergency calls. But 
there isn't yet a specification of how to connect VRS to ng911 to get an 
audio/video/RTT emergency call.

The other possibility is via Direct Video Calling using the VATRP system 
from Mitre. I guess in theory that is possible now. But it will 
currently have VRS users that use VRS Provider proprietary devices. 
Anything to support conferencing that requires extra signaling won't 
work until VRS users can actually use RUM-compliant RUEs. But that could 
work with vanilla RTT as long as the DVC device sets up the conferencing.

	Thanks,
	Paul

> Brian
> 
>> On Aug 29, 2019, at 12:20 PM, Malloy, Jim <[email-masked] 
>> <mailto:[email-masked]>> wrote:
>>
>> Do we intend for the RUE to accommodate multiple party calls (more 
>> than three)? I don't recall any language related to anything but a 
>> point to point call.  For emergency calls, conferencing in the PSAP 
>> (three parties) may be appropriate.  Are we looking for more than that?
>>
>> --Jim Malloy
>>
>> To:[email-masked] <mailto:[email-masked]>
>> Perhaps for conferences it is better to just use line-at-a-time chat. 
>> If necessary, I presume there could be gateways between RTT and chat. 
>> A RUE could have the capability to negotiate down from RTT to chat.
>>
>> Thanks,
>> Paul
>>
>>>>> <[email-masked] 
>>>>> <mailto:[email-masked]><mailto:[email-masked]>> 
>>>>> wrote:
>>>>>
>>>>>>> <[email-masked] 
>>>>>>> <mailto:[email-masked]><mailto:[email-masked]>>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>> <mailto:[email-masked]><mailto:[email-masked]>
>>>>> +46 708 204 288
>>>>
>>>>
>>> --

---

From: Brian Rosen <[email-masked]>
Date: Thu, 29 Aug 2019 14:12:45 -0400

Inline

> On Aug 29, 2019, at 2:06 PM, Paul Kyzivat <[email-masked]> wrote:
> 
> I agree conceptually. But we need to recognize that we are out ahead of VRS. Near term there is no way to get an audio/video/RTT bridge into a call with a VRS user. Because the bridge is not in the VRS iTRS it will only be able to get connected as an audio call with and interpreter
Both the user and the VRS CA become members of the conference, and they arrange for the user to see the CA in the “big window”.  We did this in the IVC work although the FCC provided the interpreters instead of a VRS service.  The bridge they used had a few issues but mostly it worked.

> 
> The first form of conference is likely to be for emergency calls. But there isn't yet a specification of how to connect VRS to ng911 to get an audio/video/RTT emergency call.
That is incorrect.  It’s completely specified in NENA-STA-010

> 
> The other possibility is via Direct Video Calling using the VATRP system from Mitre. I guess in theory that is possible now. But it will currently have VRS users that use VRS Provider proprietary devices. Anything to support conferencing that requires extra signaling won't work until VRS users can actually use RUM-compliant RUEs. But that could work with vanilla RTT as long as the DVC device sets up the conferencing.
I still don’t think that surfaces any additional requirements for the rum-compatible device.

> 
> 	Thanks,
> 	Paul
> 
>> Brian
>>> On Aug 29, 2019, at 12:20 PM, Malloy, Jim <[email-masked] <mailto:[email-masked]>> wrote:
>>> 
>>>>>> <[email-masked] <mailto:[email-masked]><mailto:[email-masked]>> wrote:
>>>>>> 
>>>>>>>> <[email-masked] <mailto:[email-masked]><mailto:[email-masked]>>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Hi,
>>>>>>>> 
>>>>>> [email-masked] <mailto:[email-masked]><mailto:[email-masked]>
>>>>>> +46 708 204 288
>>>>> 
>>>>> 
>>>> --

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 29 Aug 2019 15:02:05 -0400

> Both the user and the VRS CA become members of the conference, and they arrange for the user to see the CA in the “big window”.  We did this in the IVC work although the FCC provided the interprete…

Is this deployed and in use with actual providers on real VRS calls?

> That is incorrect.  It’s completely specified in NENA-STA-010

I haven't heard anything about this when working with providers on the 
VRS provider profile. But it doesn't show up on an interface between 
providers, so I guess there hasn't been any need for it to come up.

But because this is limited to a private interface between the providers 
and NENA it isn't applicable to any other cases.

> I still don’t think that surfaces any additional requirements for the rum-compatible device.

As long as no extra signaling is required. For instance, multiple RTT 
data channels signaled in SDP.

	Thanks,
	Paul

---

From: Brian Rosen <[email-masked]>
Date: Thu, 29 Aug 2019 15:11:00 -0400

> On Aug 29, 2019, at 3:02 PM, Paul Kyzivat <[email-masked]> wrote:
> 
Not that I know of.  It would require regulatory action dealing with compensation because the VRS CA isn’t in a 2 way call with the deaf/HoH person.  I’ve been in conferences where this was finessed by the deaf user.  He had two video devices.  He was in a VRS call with a CA who was audio dialed into the conference.  The same deaf participant was a video (and screen share) participant in the same conference.  I still don’t see any rum requirement.

> I haven't heard anything about this when working with providers on the VRS provider profile. But it doesn't show up on an interface between providers, so I guess there hasn't been any need for it to come up.
That is correct
> 
> But because this is limited to a private interface between the providers and NENA it isn't applicable to any other cases.
It directly affects the rum device.  The rue spec says the right thing (support RFC6881 and support REFER).
> As long as no extra signaling is required. For instance, multiple RTT data channels signaled in SDP.
I don’t think so.  I think we’re headed towards a single RTT channel per participant using RFC4103.  A conference would need an RTT mixer.  Every participant has a single two way RTT channel to the bridge.  

Brian

---

From: "Malloy, Jim" <[email-masked]>
Date: Thu, 29 Aug 2019 19:59:31 +0000

“line at a time” is a user interface issue – RUM is defining a machine interface.  How it’s presented to the user is out of scope.

--Jim

From: Rum <[email-masked]> On Behalf Of Brian Rosen
Sent: Thursday, August 29, 2019 1:15 PM
To: Kyzivat, Paul <[email-masked]>
Cc: [email-masked]
Subject: [EXT] Re: [Rum] Real-time text in WebRTC is discussed in mmusic - a topic closely telated to rum

Brian

On Aug 29, 2019, at 12:05 PM, Paul Kyzivat <[email-masked]<mailto:[email-masked]>> wrote:

               Thanks,
               Paul

On Aug 27, 2019, at 4:43 PM, Gunnar Hellström <[email-masked]<mailto:[email-masked]> <mailto:[email-masked]>> wrote:

Alice: I waited for you
Bob: I didn’t see you
Alice: sorry

What should happen?

-------------------From T.140---------------------------

8.2 Erase last character

  /Gunnar

Brian

On Aug 27, 2019, at 9:52 AM, Gunnar Hellström <[email-masked]<mailto:[email-masked]> <mailto:[email-masked]>> wrote:

Hi,

Regards

Gunnar

--
-----------------------------------------
Gunnar Hellström
Omnitor
[email-masked]<mailto:[email-masked]>
+46 708 204 288

--
-----------------------------------------
Gunnar Hellström
Omnitor
[email-masked]<mailto:[email-masked]> <mailto:[email-masked]>
+46 708 204 288

--
-----------------------------------------
Gunnar Hellström
Omnitor
[email-masked]<mailto:[email-masked]>
+46 708 204 288

--
Rum mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rum

---

From: Gunnar Hellström <[email-masked]>
Date: Thu, 29 Aug 2019 22:57:37 +0200

Brian said: we agreed we were using 4103 in RUM.

Good toknow. But is there not a hope to be able to even use browser 
technology for the implementation. The browsers only support RTP for 
audio and video. Can you make RTP-based RTT work in that environment and 
encrypted with the security method specified for WebRTC?

See a bit more below:

Den 2019-08-29 kl. 21:59, skrev Malloy, Jim:

> “line at a time” is a user interface issue – RUM is defining a machine 
> interface.  How it’s presented to the user is out of scope.
>
Well, the control and coding must be made so that it is possible to make 
a good presentation. Transmission only line-by-line would not suit the 
intention of RTT. It is hard to make a good RTT presentation with a 
client that is only expecting text from one other participant and a 
mixer that has the task to send text from many sources combined in one 
stream. The best result is usable but not nice. And it will be aimed at 
just one way to present RTT. No freedom to plan the presentation 
according to user preferences.

A receiver may, if there is a desire for such functionality, store 
received text until a received message is complete. I do not think that 
it is to prefer in any situation.

Regards

Gunnar

> --Jim
>
> *From:* Rum <[email-masked]> *On Behalf Of * Brian Rosen
> *Sent:* Thursday, August 29, 2019 1:15 PM
> *To:* Kyzivat, Paul <[email-masked]>
> *Cc:* [email-masked]
> *Subject:* [EXT] Re: [Rum] Real-time text in WebRTC is discussed in 
> mmusic - a topic closely telated to rum
>
> “Line at a time” could be an implementation option.  The protocol 
> mechanism is all streams head to a mixer and the mixer sends a single 
> stream to each participant.
>
>     If you have messages on your screen from multiple parties, and
>     many of them are updating in real-time, are you going to be able
>     to perceive what is going on?
>
>     3-party conversations, that doesn't scale up. With many parties,
>     some typing may scroll off the screen before it is complete.
>
>     Perhaps for conferences it is better to just use line-at-a-time
>     chat. If necessary, I presume there could be gateways between RTT
>     and chat. A RUE could have the capability to negotiate down from
>     RTT to chat.
>
>     Thanks,
>     Paul
>
>             You don’t have the problem in Instant Messaging, because
>             you can’t backspace or delete a sent message.  Of course
>             if multiple people are typing simultaneously in such
>             systems, message order will be confusing in that instant.
>
>         unexpected order. There is also at least one instant messaging
>         service that allows modification in already sent message. But
>         I think it has limitations to only accept that in the last
>         message sent. it is convenient anyway.
>
>
>             Anyway, we need to specify the mixer for RTT so it
>             receives each of the RTT streams and produces a single
>             composite stream for each participant.
>
>         multi-party text conversation on a terminal that has no
>         awareness of multi-party calls. There are some obvious
>         drawbacks, and a terminal designed with multi-party awareness
>         will be able to present multi-party call contents in a more
>         flexible way. Only two parties at a time will be allowed to
>         display added text in real-time, while the other parties’
>         produced text will need to be stored in the multi-party server
>         for a moment awaiting a suitable occasion to be displayed.
>         There are also some cases of erasure that will not be
>         performed on the target text but only indicated in another
>         way. Even with these drawbacks, the procedure provides an
>         opportunity to display text from more than two parties in a
>         smooth and readable way.
>         conference awareness, but want to see support for
>         conference-aware terminals, where text from more than two
>         parties can be presented in real-time, and the end user or app
>         can have influence over the presentation style - e.g. select
>         between the multiple column view and the
>         one-column-with-labels view.  A mixer for that case would only
>         need to assure that the receiver has the right kind of
>         multi-party awareness and send RTT text with source
>         information attached, and let the receiving terminal sort out
>         the presentation. This is already possible with CSRC and CNAME
>         when using RTP, but we lose that possibility natively when
>         using the WebRTC data channel to transport RTT, and would need
>         to specify a way to include the source also for that case.
>         By the way, what is your current view of how to transport RTT
>         for RUM, now when you say that you will use WebRTC transports
>         for media?
>         Regards
>         Gunnar
>
>
>
>                     The problem of conference 4103 RTT is high on my
>                     list of work I need to get done.  So, I’m
>                     motivated to help out.
>
>                 Thanks, great.
>
>                     The basic problem is that we’re going to get very
>                     inconsistent UI doing it that way, because of how
>                     systems will handle backspace of one party that
>                     extends beyond responses from other parties:
>
>                 (well, for me the currently most basic problem is to
>                 have a reliable way to append received text to the
>                 already presented text of the right participant. And
>                 that is getting worse in WebRTC than it was in RFC
>                 You are right that there are a number of ways to
>                 handle the RTT UI. And just as inconsistencies are
>                 common with a message oriented UI, where messages show
>                 up in a confusing order because two users completed
>                 messages in an unexpected time order, it is possible
>                 that RTT text gets displayed in a strange order after
>                 erasure and retyping. It is better for RTT than for
>                 message oriented presentation, and user get used to it
>                 in both cases.  With the labelled style in one column
>                 you have in the example, I would recommend that first
>                 5 backspaces erase "sorry", next backspace erases the
>                 line separator, and pulls down "I waited for you" to
>                 be shown last, as an uncompleted text. Then the next 6
>                 backspaces erase so that only "I waited f" is
>                 displayed. When Alice adds text and end with a new
>                 line, the corrected sentence is allowed to flow up
>                 when new text is added from any participant.  That
>                 causes a bit strange order, but it is just as
>                 manageable as when text in messaging applications
>                 appear in an unexpected order so that one message
>                 seems to be a respone on something totally else than
>                 what was intended.
>
>                 We want to keep sentences or at least phrases from
>                 each participant together in a readable unit. Already
>                 that causes a design decision on where to place the
>                 completed chunk of text once the user has completed
>                 it. The start of the chunk may be older than completed
>                 text from other participants which would motivate to
>                 move it up a bit in the presentation. But the end of
>                 it is at that moment the latest text to present. I
>                 think it is best to let the finished text be presented
>                 last on the display, but let others' newer text push
>                 Procedure: On the receiving end: Move the insertion
>                 point to the last character and erase it.
>                 Combined characters are erased as a unit, with one BS
>                 erasing the whole character even if it is
>                         A topic is currently discussed in mmusic that
>                         is closely related to rum. it is WebRTC
>                         transport of real-time text.
>
>                         The draft is
>                         draft-holmberg-mmusic-t140-usage-data-channel .
>
>                         The only issue that seems to be remaining is
>                         how to transport RTT data to and from a
>                         conference server that combines all traffic
>                         per media in a meeting in one data stream.
>                         That is not very elegantly specified for RFC
>                         4103 transport of RTT in RTP either, so we
>                         might want to do a rapid action together to
>                         solve the multi-party RTT MCU case in a
>                         general and consistent way.
>
>                         Regards
>
>                         Gunnar
>
>                         --
>                         <mailto:[email-masked]>
>                         +46 708 204 288
>
>                 --