Subject: [AVTCORE] BUNDLE and same payload type in different "bundled" m= lines

From: Iñaki Baz Castillo <[email-masked]>
Date: Tue, 8 Mar 2016 17:47:25 +0100

Hi,

I've been referred to this WG in order to report a problem when it
comes to multiple streams in a SDP using BUNDLE. Each stream has its
own m=audio/video section and they could share the same transport (so
they are "bundled") or not (it does not really matter now).

The problem is described in a thread I've started in MMUSIC WG:

  http://www.ietf.org/mail-archive/web/mmusic/current/msg16017.html

Main problem is that, according to "old" rules it seems that SSRC is
mostly ignored when it comes to identify which stream a RTP packet
belongs to. This seems to happen when following the RTCP processing
rules, and there are also many RTP/SDP devices that assume unique
payload types within the same RTP session.

So the problem I report is basically the following:

If 100 participants join the same SFU conference and all of them send
VP8 codec with payload 100 and same parameters, then each participant
will be provided with a SDP containing 100 (or 101) m=video sections
with a=sendonly, and each m=video MUST have a different payload type
(rewritten by the SFU) for the VP8 codec (as they belong to different
video streams from different participants).

...but there are just 96-127 dynamic payload values.

So in case of participants sending audio and video, should we state
that a WebRTC SFU conference can just handle (127-96+1)/2 = **16**
participants?

Thanks a lot.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 9 Mar 2016 11:36:10 +0100

Hi,

I want to clarify this a bit.

What Iñaki see as an issue that AVTCORE clearly has ownership of is that 
we consider payload types to be set on RTP session level, and not on a 
more specific level like on a per SSRC basis.

I did note that such a change would have significant impact not only on 
our specifications but also on implementation. In addition some use 
cases where you don't have or need pre-signalling of the sources, one 
need the session level configuration of PTs.

> that a WebRTC SFU conference can just handle (127-96+1)/2 = **16**

BUNDLE do allow multiple m= lines and thus media sources to use the same 
PT, given that they are identically configured. Please see Section 
10.1.1 of draft-ietf-mmusic-sdp-bundle-negotiation-27:

    o  A specific payload type value can be used in multiple bundled "m="
       lines if each codec associated with the payload type number shares
       an identical codec configuration [Section 10.1.2].

Thus, I don't think the issue is as big as Iñaki originally thought. 
Also, I want to make clear that one can use more than just the 
dynamically payload types, all PT values can be configured dynamically 
if need exists, overriding any static mappings. Thus 128 values are in 
theory available. With RTP/RTCP Multiplexing, the upper limit is around 
96 PTs.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Services, Media and Network features, Ericsson Research EAB/TXM
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Färögatan 6                 | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: Bernard Aboba <[email-masked]>
Date: Wed, 9 Mar 2016 14:29:46 -0800

On Mar 8, 2016, at 8:47 AM, Iñaki Baz Castillo <[email-masked]> wrote:
> If 100 participants join the same SFU conference and all of them send VP8 codec with payload 100 and same parameters, then each participant
> will be provided with a SDP containing 100 (or 101) m=video sections with a=sendonly, and each m=video MUST have a different payload type (rewritten by the SFU) for the VP8 codec (as they belong to different video streams from different participants).

[BA] What specific text in BUNDLE mandates this? In particular, the VP8 decoder can decode anything that the encoder can send, so the streams from the different participants do not require unique decoder settings. So AFAICT the SFU could send N streams to a given participant, each using PT=100, but with N distinct SSRCs.

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Thu, 10 Mar 2016 23:44:40 +0100

> [BA] What specific text in BUNDLE mandates this? In particular, the VP8 decoder can decode anything that the encoder can send, so the streams from the different participants do not require unique d…

At the end it happens that RFC 3550 (RTP) defines "RTP session" as all
the stuff sent by an RTP participant over a specific transport
(5-tuple), and it mandates PT to be unique within a "RTP session".

Due to that, it seems that legacy middleboxes/endpoints just take the
PT to identify codecs.

Then BUNDLE arrives and a "RTP session" no longer identifies a single
transport for a single RTP participant. But legacy systems still
assume that, so PT must remain unique.

I don't yet understand why the same PT value can be used for different
bundled streams if they refer to the same codec+configuration, since
those PTs now belong to different streams. But I don't care too much
about that.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: "Roni Even" <[email-masked]>
Date: Fri, 11 Mar 2016 00:53:14 +0200

Hi,
I assume that "legacy" endpoints will not support bundle so they will use different transport addresses for each m-line having different RTP sessions

Roni

> -----Original Message-----
> From: avt [mailto:[email-masked]] On Behalf Of I?aki Baz Castillo
> Sent: Friday, March 11, 2016 12:45 AM
> To: Bernard Aboba
> Cc: [email-masked]
> Subject: Re: [AVTCORE] BUNDLE and same payload type in different
> "bundled" m= lines
> 
> > [BA] What specific text in BUNDLE mandates this? In particular, the VP8
> decoder can decode anything that the encoder can send, so the streams
> from the different participants do not require unique decoder settings. So
> AFAICT the SFU could send N streams to a given participant, each using
> PT=100, but with N distinct SSRCs.
> 
> At the end it happens that RFC 3550 (RTP) defines "RTP session" as all the
> stuff sent by an RTP participant over a specific transport (5-tuple), and it
> mandates PT to be unique within a "RTP session".
> 
> Due to that, it seems that legacy middleboxes/endpoints just take the PT to
> identify codecs.
> 
> transport for a single RTP participant. But legacy systems still assume that, so
> PT must remain unique.
> 
> bundled streams if they refer to the same codec+configuration, since those
> PTs now belong to different streams. But I don't care too much about that.
> 
> 
> --
> Iñaki Baz Castillo
> <[email-masked]>
> 
> _______________________________________________
> Audio/Video Transport Core Maintenance
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 11 Mar 2016 00:03:49 +0100

> I assume that "legacy" endpoints will not support bundle so they will use different transport addresses for each m-line having different RTP sessions

I don't even assume those "legacy" endpoints will support rtcp-mux,
and AFAIK Chrome and Firefox are about removing "no rtcp-mux"...

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 11 Mar 2016 09:50:58 +0100

Den 2016-03-10 kl. 23:44, skrev Iñaki Baz Castillo:
>> [BA] What specific text in BUNDLE mandates this? In particular, the
>> VP8 decoder can decode anything that the encoder can send, so the
>> streams from the different participants do not require unique
>> decoder settings. So AFAICT the SFU could send N streams to a given
>> participant, each using PT=100, but with N distinct SSRCs.
>
> At the end it happens that RFC 3550 (RTP) defines "RTP session" as
> all the stuff sent by an RTP participant over a specific transport
> Due to that, it seems that legacy middleboxes/endpoints just take
> the PT to identify codecs.
>
> Then BUNDLE arrives and a "RTP session" no longer identifies a
> single transport for a single RTP participant. But legacy systems
> still assume that, so PT must remain unique.

I don't really agree to this characterization of Bundle. Bundle is SDP
solution for how one adds media source level information around the RTP
streams that all are sent within a single RTP session.

In non offer/answer usage of SDP, like in SAP and RTSP the m= lines
represents a RTP session, not a media source. In these cases you don't
have any information about the specific RTP streams, there is an
assumption that there will be sent one or more per direction in these
usages.

>
> I don't yet understand why the same PT value can be used for
> different bundled streams if they refer to the same
> codec+configuration, since those PTs now belong to different streams.
> But I don't care too much about that.
>

To be clear, from a high level it exists a possibility to define usages
of the RTP that would define the PT value to be scoped on SSRC basis,
rather then RTP session level. However the devil is in the details and
such a usage would require careful review of existing specifications to
find where there might be issues. I am especially worried about issues
in the RTCP parts, for cases where PT information are used. Normally
this is clearly done in the context of an SSRC. But, to be certain there
are no issues one has to review things.  I am also expecting that more
than one RTP stack implementation are unable to support this without
modifications, possibly extensive ones. This as the data structures for
the PT are associated with the RTP session rather than individual SSRCs.

I want to be clear that if one like to pursue this idea of redefining 
what PT are scoped by, then this requires AVTCORE work to create a 
specification for such a usage. If that is a new profile or something 
else that will have to be for future discussions.

So, I would like to make sure that we are on a common understanding what 
a SFU have to do with the RTP streams to make it possible to re-use the PTs.

So if we start with A and B communicating across an SFU. I will continue 
this from your example when you raised this in MMUSIC:

1) mid: alice-audio,  codec: opus,  payload: 100, ssrc: 1111
2) mid: bob-audio,   codec: g722,  payload: 100, ssrc: 2222

This would mean that Carol receives an SDP with two "m=audio"
sections using BUNDLE:

   m=audio [...] 100
   a=sendonly
   a=mid:alice-audio
   a=rtpmap:100 opus/48000/2
   a=ssrc:1111 cname:foo
   [...]
   m=audio [...] 100
   a=sendonly
   a=mid:bob-audio
   a=rtpmap:100 g722/48000
   a=ssrc:2222 cname:bar
   [...]

So from my perspective what the conference signalling server has to send 
Carol is an SDP that looks like this:

This would mean that Carol receives an SDP with two "m=audio"
sections using BUNDLE:

   m=audio [...] 100
   a=sendonly
   a=mid:alice-audio
   a=rtpmap:100 opus/48000/2
   a=ssrc:1111 cname:foo
   [...]
   m=audio [...] 101
   a=sendonly
   a=mid:bob-audio
   a=rtpmap:101 g722/48000
   a=ssrc:2222 cname:bar
   [...]

Then the SFU will need to change all RTP packets with PT=100 from Bob to 
PT=101 when forwarding them to Carol. The packets from Alice it can 
leave unchanged from an PT perspective.

If David joins the session and can do both codecs and offers like this:

m=audio [...] 98 99
a=sendrecv
a=rtpmap:98 opus/48000/2
a=rtpmap:99 g722/48000
a=ssrc:4444 cname:groo

Then I expect the conference server to determine that these 
configurations are identical to already existing ones and answer:

m=audio [...] 98
a=sendonly
a=mid:alice-audio
a=rtpmap:98 opus/48000/2
a=ssrc:1111 cname:foo
[...]
m=audio [...] 99
a=sendonly
a=mid:bob-audio
a=rtpmap:99 g722/48000
a=ssrc:2222 cname:bar
[...]
m=audio [...] 98 99
a=sendonly
a=mid:carol-audio
a=rtpmap:98 opus/48000/2
a=rtpmap:99 g722/48000
a=ssrc:3333 cname:skog
[...]
m=audio [...] 100 101
i=M block for David's audio stream
a=recv
a=rtpmap:100 opus/48000/2
a=rtpmap:101 g722/48000

So the SFU will translate the PTs towards Dave, to match what Dave 
accepts receiving. The SFU request Dave's endpoint to use the PTs it has 
defined as its common reception table out of the ones the Dave support.

Yes, using RTP middleboxes and SDP offer/answer do result in that one 
will have to specific PT values in use on each leg. It is very difficult 
to get this aligned to between the legs.

Your proposal is one way to avoid that translation. Which would require 
certain changes and issues. Another possibility would be to try to 
address it in the signalling system. Because having the possibility for 
the central controller to dictate the actual configuration within the 
capabilities of the endpoints would simplify certain things. 
Unfortunately both have issues to resolve and significant legacy inertia.

Cheers

Magnus Westerlund

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 11 Mar 2016 21:28:22 +0100

> assumption that there will be sent one or more per direction in these

Not sure if I get it. At the end, in WebRTC each m= line means a
MediaStreamTrack to be sent and/or a MediaStreamTrack to be received.

> the PT are associated with the RTP session rather than individual SSRCs.

Useful information. However I will start another thread (not sure in
which mailing list) with a simple question:

As BUNDLE spec states, two bundled m= lines can just share the same PT
value if it points to the same codec name and codec configuration.
Where is specified what "same configuration" means? I expect it
depends on each codec...

> So, I would like to make sure that we are on a common understanding what a
> SFU have to do with the RTP streams to make it possible to re-use the PTs.
>
> PT=101 when forwarding them to Carol. The packets from Alice it can leave
> unchanged from an PT perspective.
>
> Then I expect the conference server to determine that these configurations
> are identical to already existing ones and answer:
>
> m=audio [...] 98
> a=sendonly
> a=mid:alice-audio
> a=rtpmap:98 opus/48000/2
> a=ssrc:1111 cname:foo
> [...]
> m=audio [...] 99
> a=sendonly
> a=mid:bob-audio
> a=rtpmap:99 g722/48000
> a=ssrc:2222 cname:bar
> [...]
> m=audio [...] 98 99
> a=sendonly
> a=mid:carol-audio
> a=rtpmap:98 opus/48000/2
> a=rtpmap:99 g722/48000
> a=ssrc:3333 cname:skog
> [...]
> m=audio [...] 100 101
> So the SFU will translate the PTs towards Dave, to match what Dave accepts
> receiving. The SFU request Dave's endpoint to use the PTs it has defined as
> its common reception table out of the ones the Dave support.

Thanks a lot. Very valuable information.

However I plan to do something a bit different:

When a new participant joins the SFU room and sends its SDP offer, the
room will check the offered codecs. If same codec+configuration
already exists in the room (with an assigned unique PT) the SFU will
generate a SDP answer including the PT in the room for that
codec+configuration. According to SDP O/A, the offerer is mandated to
send RTP using the PT indicated in the SDP answer.

With this, I will avoid PT mangling in the SFU.

> Yes, using RTP middleboxes and SDP offer/answer do result in that one will
> have to specific PT values in use on each leg. It is very difficult to get
> this aligned to between the legs.
>
> certain changes and issues. Another possibility would be to try to address
> it in the signalling system. Because having the possibility for the central
> controller to dictate the actual configuration within the capabilities of
> the endpoints would simplify certain things. Unfortunately both have issues
> to resolve and significant legacy inertia.

That's exactly what I propose above :)
However I do know that, probably, I will have to implement PT mangling
in some cases (for example when the SFU sends a re-offer to a browser
indicating its preferred PT values but the browser replies with
different values).

Thanks a lot Magnus.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 16 Mar 2016 13:12:16 +0000

Hi,

>> I assume that "legacy" endpoints will not support bundle so they will 
>> use different transport addresses for each m-line having different RTP 
>> sessions
>

That is not true. There are legacy endpoints supporting rtcp-mux.

Regards,

Christer