Subject: [AVTCORE] Multi-party real-time text requiring new RTP payload type

From: Gunnar Hellström <[email-masked]>
Date: Tue, 21 Apr 2020 11:18:14 +0200

Hi,

I have got an off-list comment that the proposed use of the CSRC-list 
members to indicate source of the primary and the different redundancy 
generations of the T140blocks would require registration of a new 
payload type instead of saying that it is a refined use of text/red 
updated from RFC 4103.

( this is about 
https://www.ietf.org/id/draft-hellstrom-avtcore-multi-party-rtt-source-03.txt 
)

The purpose is to speed up the switch of source of text from the mixer 
so that every packet can have primary text from another source than the 
previous packet. Without the update, the mixer would need to send two 
packets with just redundant text before sending new text from another 
source.

I am not yet totally convinced that we need to introduce a new payload 
type, but find it possible.

Let us call it "text/rex" for "redundancy extended".

It would cause a quite straightforward sdp negotiation of parties 
supporting both the traditional text/red format from RFC 4103, and the 
updated text/rex.

Offer example :

       m=text 11000 RTP/AVP 101 100 98
       a=rtpmap:98 t140/1000
       a=rtpmap:100 red/1000
       a=rtpmap:101 rex/1000
       a=fmtp:100 98/98/98
       a=fmtp:101 98/98/98
       a=rtt-mix

  Answer example  from a multi-party capable device
       m=text 11000 RTP/AVP 101 98
       a=rtpmap:98 t140/1000
       a=rtpmap:101 rex/1000
       a=fmtp:101 98/98/98
       a=rtt-mix

Answer example from a multi-party unaware device:

       m=text 11000 RTP/AVP 100 98
       a=rtpmap:98 t140/1000
       a=rtpmap:100 red/1000
       a=fmtp:100 98/98/98

It may look strange that the attribute a=rtt-mix is still there as an 
indication of multi-party capability when that already is implied in the 
"rex" payload type. But it is for the very rare case that the network 
conditions would allow use of bare t140 payload type without redundancy. 
Then there is a need to indicate multi-party awareness by a=rtt-mix .

If we accept this move, we could also open for discussion if the format 
proposed by James Hamlin earlier in this list, where the maximum number 
of 16 members in the CSRC list allows transport of text from up to 16 
sources in the same packet. That would make the mixing of real-time text 
as efficient and rapid as that of audio and video. A question is though 
if current implementations are well behaving and do not collapse by 
getting the combined sdp.

If we do not accept this move and instead base the multi-party mixing on 
the current text/red format with short transmission intervals, we can 
claim that simultaneous typing is quite rare and that the introduced 
delay of the first few characters of about 200 ms from a second source 
when one is already sending is acceptable, knowing that most text 
conversations are at speeds of under 12 characters per second and that 
the transmissions from each source anyway comes in 300 ms intervals.  
Some jerkiness and delay will be noticable when there is more than two 
simultaneous senders.

Regards

Gunnar

-- 
Gunnar Hellström
GHAccess
[email-masked]

---

From: Brian Rosen <[email-masked]>
Date: Tue, 21 Apr 2020 14:56:14 -0400

I’m okay with the new payload type.  All of this would be new code, so dealing with the new payload type wouldn’t be confusing or difficult. 
I’m not aware of any implementations of t140 without redundancy, but I’m also not bothered by including rtt-mix.

I’m also okay with James Hamlin’s proposal.  I like the effect enough that I’m not going to worry about bad implementations.  Again, supporting this is new code, so if your implementation of T140 and SDP in general is lacking, this would be a good time to fix that.

Brian

> On Apr 21, 2020, at 5:18 AM, Gunnar Hellström <[email-masked]> wrote:
> 
> Hi,
> 
> I have got an off-list comment that the proposed use of the CSRC-list members to indicate source of the primary and the different redundancy generations of the T140blocks would require registration of a new payload type instead of saying that it is a refined use of text/red updated from RFC 4103.
> 
> ( this is about https://www.ietf.org/id/draft-hellstrom-avtcore-multi-party-rtt-source-03.txt <https://www.ietf.org/id/draft-hellstrom-avtcore-multi-party-rtt-source-03.txt> )
> 
> The purpose is to speed up the switch of source of text from the mixer so that every packet can have primary text from another source than the previous packet. Without the update, the mixer would need to send two packets with just redundant text before sending new text from another source.
> 
> I am not yet totally convinced that we need to introduce a new payload type, but find it possible. 
> 
> It would cause a quite straightforward sdp negotiation of parties supporting both the traditional text/red format from RFC 4103, and the updated text/rex. 
> 
> Offer example :
> 
> It may look strange that the attribute a=rtt-mix is still there as an indication of multi-party capability when that already is implied in the "rex" payload type. But it is for the very rare case that the network conditions would allow use of bare t140 payload type without redundancy. Then there is a need to indicate multi-party awareness by a=rtt-mix .
> 
> If we accept this move, we could also open for discussion if the format proposed by James Hamlin earlier in this list, where the maximum number of 16 members in the CSRC list allows transport of text from up to 16 sources in the same packet. That would make the mixing of real-time text as efficient and rapid as that of audio and video. A question is though if current implementations are well behaving and do not collapse by getting the combined sdp.
> 
> If we do not accept this move and instead base the multi-party mixing on the current text/red format with short transmission intervals, we can claim that simultaneous typing is quite rare and that the introduced delay of the first few characters of about 200 ms from a second source when one is already sending is acceptable, knowing that most text conversations are at speeds of under 12 characters per second and that the transmissions from each source anyway comes in 300 ms intervals.  Some jerkiness and delay will be noticable when there is more than two simultaneous senders.
> 
> Regards
> 
> Gunnar 
> 
> -- 
> Gunnar Hellström
> GHAccess
> [email-masked] <mailto:[email-masked]>_______________________________________________
> Audio/Video Transport Core Maintenance
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt

---

From: Gunnar Hellström <[email-masked]>
Date: Wed, 22 Apr 2020 22:01:27 +0200

Thanks Brian,

Den 2020-04-21 kl. 20:56, skrev Brian Rosen:
> I’m okay with the new payload type.  All of this would be new code, so 
> dealing with the new payload type wouldn’t be confusing or difficult.
Right. And a lot from RFC 4103 is still valid with the new payload type, 
so that it can be an update to RFC 4103.
> I’m not aware of any implementations of t140 without redundancy, but 
> I’m also not bothered by including rtt-mix.
Right. No one using plain T.140 without redundancy currently.
>
> I’m also okay with James Hamlin’s proposal.  I like the effect enough 
> that I’m not going to worry about bad implementations.
I aim at including this new payload type in next version. I also intend 
to look at what sdp would look like if we include negotiation between 
RTP and SRTP.
> Again, supporting this is new code, so if your implementation of T140 
> and SDP in general is lacking, this would be a good time to fix that.

> Brian
Thanks,

Gunnar

>
>> On Apr 21, 2020, at 5:18 AM, Gunnar Hellström 
>> <[email-masked] <mailto:[email-masked]>> 
>> wrote:
>>
>> Hi,
>>
>> members to indicate source of the primary and the different 
>> redundancy generations of the T140blocks would require registration 
>> of a new payload type instead of saying that it is a refined use of 
>> text/red updated from RFC 4103.
>>
>> ( this is about 
>> The purpose is to speed up the switch of source of text from the 
>> mixer so that every packet can have primary text from another source 
>> than the previous packet. Without the update, the mixer would need to 
>> send two packets with just redundant text before sending new text 
>> from another source.
>>
>> supporting both the traditional text/red format from RFC 4103, and 
>> the updated text/rex.
>>
>> Offer example :
>>
>> indication of multi-party capability when that already is implied in 
>> the "rex" payload type. But it is for the very rare case that the 
>> network conditions would allow use of bare t140 payload type without 
>> redundancy. Then there is a need to indicate multi-party awareness by 
>> a=rtt-mix .
>>
>> If we accept this move, we could also open for discussion if the 
>> format proposed by James Hamlin earlier in this list, where the 
>> maximum number of 16 members in the CSRC list allows transport of 
>> text from up to 16 sources in the same packet. That would make the 
>> mixing of real-time text as efficient and rapid as that of audio and 
>> video. A question is though if current implementations are well 
>> behaving and do not collapse by getting the combined sdp.
>>
>> If we do not accept this move and instead base the multi-party mixing 
>> on the current text/red format with short transmission intervals, we 
>> can claim that simultaneous typing is quite rare and that the 
>> introduced delay of the first few characters of about 200 ms from a 
>> second source when one is already sending is acceptable, knowing that 
>> most text conversations are at speeds of under 12 characters per 
>> second and that the transmissions from each source anyway comes in 
>> 300 ms intervals.  Some jerkiness and delay will be noticable when 
>> there is more than two simultaneous senders.
>>
>> Regards
>>
>> Gunnar
>>
>> -- 
>> Gunnar Hellström
>> GHAccess
>> _______________________________________________
>> [email-masked] <mailto:[email-masked]>

---

From: [email-masked] (Dale R. Worley)
Date: Wed, 22 Apr 2020 22:27:07 -0400

I assume that there's a good reason why the case of redundant
transmission of multi-party text isn't done by suitably interleaving the
redundant packets of the serveral text sources (which can be separated
by their SSRC values).  I can't remember if this is a violation of the
RTP rules, or perhaps it's just inefficient.  But a quick read of the
draft didn't mention why this approach doesn't work, and it seems like
it would be a useful note to the reader.

Dale

---

From: Gunnar Hellström <[email-masked]>
Date: Thu, 23 Apr 2020 10:36:51 +0200

Hi Dale,

Thanks, a good question. It is the right time to consider which source 
multiplexing and reliability method to select, now when we need to 
introduce a new negotiation.

More below

> it would be a useful note to the reader.
Draft-hellstrom-avtcore-multi-party-rtt-source is intended to contain 
the solution for the type of bridge used in NG9-1-1 and 3GPP IMS, which 
seems to be based on the RFC4535 SIP centralised conference framework.
Backgrounds and discussions of possible methods with a wider scope are 
collected in

draft-hellstrom-avtcore-multi-party-rtt-solutions
<https://datatracker.ietf.org/doc/draft-hellstrom-avtcore-multi-party-rtt-solutions/?include_text=1>

where section 4.2.1 is similar to what you propose with separate ssrc:s 
per source. Each discussed method also has pros and cons briefly 
documented. Having separate SSRC per source is what RTP calls using a 
RTP translator.

Yes, I would have liked to use that method. But we have an urgency in 
implementing the selected method in NG9-1-1 systems and in 3GPP IMS. And 
it is said that many current RTP implementations do not support multiple 
SSRCs in the same RTP session and even that RFC 3264 says that one 
m-section in sdp can only support one SSRC. However, RFC 4588 (FEC) 
section 8.8 shows an SSRC-multiplexing sdp example where the original 
media stream has one SSRC and the redundancy another within the same 
m-section. But that is of course no evidence that current RTP 
implementations support it.

RFC 8108 clarifies in section 3.3 that one of its applications is for 
combining multiple media sources of the same kind in the same RTP 
session. But it has the total goal of multiplexing also multiple media 
in the same RTP session. The corresponding sdp specification (with a 
much wider scope than we need) is 
draft-ietf-mmusic-sdp-bundle-negotiation and it is not supported by many 
current RTP implementations ( but maybe soon is because of its use in 
WebRTC).

Methods that could be considered but are not yet documented in 
draft-hellstrom-avtcore-multi-party-rtt-solutions 
<https://datatracker.ietf.org/doc/draft-hellstrom-avtcore-multi-party-rtt-solutions/?include_text=1> 
are (excuse me if some look far fetched):

RFC 4588 FEC, using SSRC multiplexing if that works or session 
multiplexing if needed. In this case plain T.140 RTP would be sent from 
the mixer in one stream, with the ssrc of the mixer and csrc of the 
source, and in the redundancy stream the packets would be repeated 
(twice) with another ssrc in a small redundancy header.

RFC 8122 Comedia to improve reliability, but then I have not studied how 
to arrange for multi-party use.

draft-ietf-mmusic-t140-usage-data-channel using one WebRTC data channel 
per source.

RFC 8108 section 3.3 and draft-ietf-mmusic-sdp-bundle-negotiation

Avoiding solutions because of imagined limitations in current 
implementations may be a bad design criterium, so do you suggest that we 
rapidly look further into any of these or other options?

Thanks

Gunnar

---

From: Gunnar Hellström <[email-masked]>
Date: Thu, 23 Apr 2020 18:59:34 +0200

Hi Dale,

Here is a bit more on the use of multiple SSRC:s in an RTP session.

https://mailarchive.ietf.org/arch/msg/avt/l1_r_4O4CYbjA_A9dPGX5vyEJ3Y/

In that old discussion, there was no idea proposed for how to signal a 
new participant in the conference. But there were multiple people 
preferring a solution with multiple SSRCs in the same RTP session. 
Similarly in this new discussion.

There is a possibility to signal a new SSRC for the case that the RFC 
4535 - 4575 - 4579 family for SIP centralised conference is used.  Then 
the RFC 4575 SIP Notify sent by the focus can be specified to contain a 
text media member, with a src-id element containing the SSRC of the 
added participant. So, if current RTP implementations have an API that 
tells it to accept a new stream with SSRC=x then a mechanism with 
multiple SSRCs might be used. Do they have that?

For simplicitly, the Payload Type numbers should be the same for all 
streams, otherwise there would need to be reInvites for every new 
participant, adding PT numbers to the m-line.

I am afraid that we get into too many uncertain areas just as we did 
when discussing the same topic in 2011, so that we should return to the 
single stream solution.

Regards

Gunnar

Den 2020-04-23 kl. 10:36, skrev Gunnar Hellström:
> the solution for the type of bridge used in NG9-1-1 and 3GPP IMS, 
> which seems to be based on the RFC4535 SIP centralised conference 
> framework.
> where section 4.2.1 is similar to what you propose with separate 
> ssrc:s per source. Each discussed method also has pros and cons 
> briefly documented. Having separate SSRC per source is what RTP calls 
> using a RTP translator.
>
> implementing the selected method in NG9-1-1 systems and in 3GPP IMS. 
> And it is said that many current RTP implementations do not support 
> multiple SSRCs in the same RTP session and even that RFC 3264 says 
> that one m-section in sdp can only support one SSRC. However, RFC 4588 
> (FEC) section 8.8 shows an SSRC-multiplexing sdp example where the 
> original media stream has one SSRC and the redundancy another within 
> the same m-section. But that is of course no evidence that current RTP 
> draft-ietf-mmusic-sdp-bundle-negotiation and it is not supported by 
> many current RTP implementations ( but maybe soon is because of its 
> use in WebRTC).
>
> multiplexing if needed. In this case plain T.140 RTP would be sent 
> from the mixer in one stream, with the ssrc of the mixer and csrc of 
> the source, and in the redundancy stream the packets would be repeated 
> RFC 8122 Comedia to improve reliability, but then I have not studied 
> how to arrange for multi-party use.
>
> draft-ietf-mmusic-t140-usage-data-channel using one WebRTC data 
> channel per source.
>
> implementations may be a bad design criterium, so do you suggest that 
> we rapidly look further into any of these or other options?
>
> Thanks
>
> Gunnar
>
>
>> Dale
> -- 
> Gunnar Hellström
> GHAccess

---

From: [email-masked] (Dale R. Worley)
Date: Thu, 23 Apr 2020 21:22:48 -0400

> introduce a new negotiation.

My idea wasn't that I had any better idea, but rather that there were
some obvious ideas, and clearly, people who had studied the matter found
that the obvious ideas were not the best.  What I really meant was that
it would be useful to give brief documentation why the obvious ideas
aren't best.

Dale

---

From: Gunnar Hellström <[email-masked]>
Date: Fri, 24 Apr 2020 10:55:04 +0200

Den 2020-04-24 kl. 03:22, skrev Dale R. Worley:
> Gunnar Hellstrom <[email-masked]> writes:

Your question provided a good opportunity to review solution alternatives.

Since the effort to specify multi-party real-time text aims at additions 
to current implementations of bridges and endpoints I think one 
requirement should be to keep the solution as close to the current 
solutions for the other media handles by these bridges and endpoints.

NG9-1-1, NG 112 and 3GPP CONF services specify use of the RFC 4535 SIP 
conference framework. Adding new participants in that framework usually 
does not cause any new sdp interaction with the other participants, as 
long as no new media is added. Just a notification is sent. I assume 
that it can be good to keep that way of action also for adding 
participants with real-time text. Using the RTP mixer model adding just 
members in the CSRC lists is then the most simple model. So, this is yet 
another reason for why to keep to the RTP-mixer solution for the first 
specification in the draft-hellstrom-avtcore-multi-party-rtt-source 
draft. It might be followed by other selected cases in the 
draft-hellstrom-avtcore-multi-party-rtt-solutions draft, where WebRTC 
and use of sdp-bundle can be spcified, which are out of scope for the 
first case.

Even if different solutions are discussed in the 
draft-hellstrom-avtcore-multi-party-rtt-solutions draft, I accept your 
point and will include a brief motivation also in the 
draft-hellstrom-avtcore-multi-party-rtt-source draft.

Comments are very welcome about the reasoning.

Thanks

Gunnar