Subject: [MMUSIC] RTP demux in JSEP and BUNDLE

From: Colin Perkins <[email-masked]>
Date: Thu, 8 Dec 2016 10:54:41 +0000

Hi,

[cc’ing RTCWEB and MMUSIC, since this relates to both JSEP and BUNDLE drafts]

There’s been a lot of discussion on the lists, and at the meeting in Seoul, around how RTP streams are mapped onto higher-level, application meaningful, semantic roles. In particular, around how RTP streams map onto JSEP objects for WebRTC. Magnus Westerlund and I would like to propose the following updates to JSEP and BUNDLE to try to clarify the behaviour. 

Comments and feedback very welcome.

Cheers,
Colin

# Replacement for Section 6 of draft-ietf-rtcweb-jsep-17

 <section title="Processing RTP/RTCP" anchor="sec.rtp.demux">
    <t>As described in <xref target="RFC3550"/>, RTP packets are 
    associated with RTP streams <xref target="RFC7656"/>. Metadata
    about those streams, including source description information
    and reception quality feedback is conveyed in RTCP packets.
    Each RTP stream is identified by an SSRC, and each RTP packet
    carries an SSRC value that is used to associate the packet with
    the correct RTP stream. RTCP packets also use SSRCs to identify
    the RTP streams that the reports and metadata relate to.  RTCP 
    packets generally carry multiple SSRC values and report on, or
    deliver source description information relating to, several RTP 
    streams.</t>

    <t>Each incoming RTP stream, identified by its SSRC, is mapped to 
    an m= section in the SDP. The SDP m= sections then correspond to 
    RtpReceiver objects. This allows each RTP stream to be associated 
    with an RtpTransceiver. Further processing of the RTP stream can 
    then be done at the RtpTransceiver level.  This includes using
    RID <xref target="I-D.ietf-mmusic-rid"/> to distinguish between
    multiple Encoded Streams, as well as determine which Source RTP
    stream should be repaired by a given Redundancy RTP stream.</t>

    <t>The process of mapping RTP streams onto m= sections depends on
    whether streams are bundled or not. If the SDP BUNDLE extension
    is in use, then RTP streams are mapped onto m= sections based on
    the MID values as described in 
    <xref target="I-D.ietf-mmusic-sdp-bundle-negotiation"/>.  If the
    SDP BUNDLE extension is not in use, each m= section corresponds
    to a transport layer connection and the RTP streams received on
    that connection correspond to the m= section.</t>

    <t>Incoming RTCP packets contain metadata including reception
    quality feedback, source description information, and other
    signalling relating to RTP streams. The RTCP packets are parsed,
    the associated RTP streams are identified based on the included
    SSRC values, and the metadata relating to those RTP streams is
    updated (this might include updating the MID information, used
    to associate RTP streams with m= sections, if the SDP BUNDLE
    extension is in use). This updated metadata is available to the
    RtpTransceiver objects associated with those RTP streams.
    </t>
 </section>

# Replacement text for section 10.2 of draft-ietf-mmusic-sdp-bundle-negotiation-36

     <section anchor="sec-rtp-pt"
              title="Associating RTP Streams With Correct SDP Media Description"
              toc="default">
       <t>As described in <xref format="default" pageno="false"
       target="RFC3550"/>, RTP packets are associated with RTP streams <xref
       format="default" pageno="false" target="RFC7656"/>. Each RTP stream is
       identified by an SSRC value, and each RTP packet carries an SSRC value
       that is used to associate the packet with the correct RTP stream. RTCP
       packets also uses SSRCs to identify on which RTP streams any report or
       feedback relate to. Thus, an RTCP packet will commonly carry multiple
       SSRC values, and might therefore be providing feedback or report on
       multiple RTP streams. </t>

       <t>In order to be able to process received RTP packets correctly it
       must be possible to associate an RTP stream with the correct "m="
       line, as the "m=" line and SDP attributes associated with the "m="
       line contain information needed to process the packets.</t>

       <t>As all RTP streams associated with a BUNDLE group are part of the
       same RTP session and using the same address:port combination for
       sending and receiving RTP/RTCP packets, the local address:port
       combination cannot be used to associate an RTP stream with the correct
       "m=" line. In addition, multiple RTP streams might be associated with
       the same "m=" line.</t>

       <t>Also, as described in <xref format="default" pageno="false"
       target="sec-rtp-sessions-pt"/>, the same payload type value might be
       used by multiple RTP streams, in which case the payload type value
       cannot be used to associate an RTP stream with the correct "m=" line.
       However, there are cases where each "m=" line has unique payload type
       values, and then the payload type could serve as hint to the relevant
		    "m=" line the RTP stream is associated with.</t>

       <t>An offerer and answerer can inform each other which SSRC values
       they will use for an RTP stream by using the SDP 'ssrc' attribute
       <xref format="default" pageno="false" target="RFC5576"/>. However, an
       offerer will not know which SSRC values the answerer will use until
       the offerer has received the answer providing that information. Due to
       this, before the offerer has received the answer, the offerer will not
       be able to associate an RTP stream with the correct "m=" line using
       the SSRC value associated with the RTP stream. In addition, the
       offerer and answerer may start using new SSRC values mid-session,
       without informing each other using the SDP 'ssrc' attribute.</t>

       <t>In order for an offerer and answerer to always be able to associate
       an RTP stream with the correct "m=" line, the offerer and answerer
       using the BUNDLE extension MUST support the mechanism defined in <xref
       format="default" pageno="false" target="sec-receiver-id"/>, where the
       offerer and answerer includes the identification-tag (provided by the
       remote peer) associated with an "m=" line in the RTP Streams and in 
       RTCP SDES packets part of a BUNDLE group.</t>

       <t>The mapping from an SSRC to an identification-tag is carried in
       RTCP SDES packets or in RTP header extensions (<xref format="default"
       pageno="false" target="sec-receiver-id"/>). Since a compound RTCP
       packet can contain multiple RTCP SDES packets, and each RTCP SDES
       packet can contain multiple chunks, an RTCP packet can contain several
       SSRC to identification-tag mappings. The offerer and answerer maintain
       tables mapping RTP streams identified by SSRC to "m=" lines identified 
       by the identification-tag. 
       When receiving an RTP packet carrying a MID header extension
       with the identification-tag, or an RTCP packet carrying one or
       more SDES MID items, the offerer or answerer creates a mapping
       table entry between the SSRC value and the identification-tag,
       in order to associate the RTP stream associated with that SSRC
       value with the "m=" line corresponding to the identification-tag.</t>

       <t>The mapping between the SSRC an identification-tag might change
       mid-session if, for a given SSRC value, a different identification-tag
       is provided in an RTP or RTCP packet. In that case these tables are
       updated each time an RTP/RTCP packet containing a new mappings from
       SSRC to identification-tag is received. Some considerations for
       avoiding update flaps are provided in Section 4.2.6 of <xref
       target="RFC7941"/> which should be followed. </t>

       <t>If an offerer and answerer is not able to associate an RTP stream
       with an "m=" line (using the mechanisms described in this section, or
       using other appropriate mechanism, e.g., based on the payload type
       value if it is unique to a single "m=" line), it MUST either drop the
       RTP packets associated with the RTP stream, or process them in an
       application specific manner, once non-stream specific processing
       (e.g., related to congestion control) of the RTP packets have
       occurred.</t>

       <t>When compound RTCP packets are received, they are split
       into their component RTCP packets and those component RTCP
       packets are processed based on their RTCP packet type, in
       the order in which they were placed into the compound RTCP
       packet. Non-compound RTCP packets are processed based on 
       their RTCP packet type, in the order they are received. Information
       in each RTCP packet can relate to one or more RTP streams.
       For example, RTCP Sender Report (SR) and Receiver Report (RR) 
       packets include an SSRC of sender field that indicates the
       identity of the participant that sent the RTCP packet, along
       with a list of Report Blocks. Each report contains data on the
       reception quality of a single RTP stream, identified by SSRC,
       as received by the SSRC that sent the RTCP packet. Other RTCP
       packet types similarly contain references to the SSRC of the
       sender of the RTCP packet, and the RTP streams to which it
       refers.</t>

       <t>It should always be possible to process RTCP packets, and
       store the received information in a data structure associated
       with an RTP stream, identified by SSRC, for later access and 
       use. It is possible that RTCP packets relating to an SSRC can
       be received before RTP packets relating to that SSRC, so the
       data structures relating to an SSRC might need to be created
       before the corresponding RTP stream is received.</t>

       <t>Similarly, information relating to an RTP stream might be
       received before the data needed to map it onto an m= line is
       received. Information carried in RTCP packets relating to such
       an RTP stream that is application and/or "m=" line dependent 
       MAY be dropped until the SSRCs is associated with a particular 
       "m=" line. However, information to generate RTCP report blocks
       and other basic transport level feedback or reporting needs to
       be retained, so RTCP reports relating to the stream can be 
       generated.</t>

     </section>

-- 
Colin Perkins
https://csperkins.org/

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 9 Dec 2016 10:10:20 -0700

In general, I think it makes sense to put most the details in bundle and overview in jsep as you have proposed here. 

I believe we agreed that if the PT was unique, then that would be used for the demux as well. I'd like to see that explicitly spelled out in this text instead of just left as an option allowed but not really specified by this text.  I'd also like it be clear that MID takes priority over PT. 

I prefer the much more explicit algorithm particularly for the RTCP handling as implementors have a hard time figuring out wha they need to do to process the RTCP. 

> On Dec 8, 2016, at 3:54 AM, Colin Perkins <[email-masked]> wrote:
> 
> Hi,
>

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 9 Dec 2016 07:13:36 -1000

I agree with Cullen that the algorithm structure used in current JSEP is
the better structure.

Magnus, Colin, do you think you could rewrite your text in that structure?

-Ekr

On Fri, Dec 9, 2016 at 7:10 AM, Cullen Jennings <[email-masked]> wrote:

>
> In general, I think it makes sense to put most the details in bundle and
> overview in jsep as you have proposed here.
>
> I believe we agreed that if the PT was unique, then that would be used for
> the demux as well. I'd like to see that explicitly spelled out in this text
> instead of just left as an option allowed but not really specified by this
> text.  I'd also like it be clear that MID takes priority over PT.
>
> I prefer the much more explicit algorithm particularly for the RTCP
> handling as implementors have a hard time figuring out wha they need to do
> to process the RTCP.
>
>
>
> > There’s been a lot of discussion on the lists, and at the meeting in
> Seoul, around how RTP streams are mapped onto higher-level, application
> meaningful, semantic roles. In particular, around how RTP streams map onto
> JSEP objects for WebRTC. Magnus Westerlund and I would like to propose the
> following updates to JSEP and BUNDLE to try to clarify the behaviour.
> >
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>

---

From: Bernard Aboba <[email-masked]>
Date: Fri, 9 Dec 2016 10:01:43 -0800

I agree with Eric and Cullen that a more explicit algorithm is needed.  The
proposed text is not specific enough to resolve observed implementation
differences.

Cullen said:

" I'd also like it be clear that MID takes priority over PT."

[BA] Yes.  This is something that multiple implementations appear to agree
on.  Also, Peter's suggestion that a MID mismatch result in a packet drop
should be restored.

"I believe we agreed that if the PT was unique, then it would be used for
demux as well."

[BA] That is my understanding as well.

However, I'd also like to understand what happens if the PT is not unique.
The proposed text does not provide enough detail and existing
implementations differ in how they treat non-unique PTs. I am familiar with
an implementation that declares SDP with non-unique PTs to be in error.
Another implementation fills a Payload Type table with a single value, with
the value selected depending on whether SSRCs are also specified (e.g.
specification of SSRCs is taken as an indication that only SSRC matching is
desired, and therefore that a Payload Type entry is not needed).  The ORTC
API spec says to latch the SSRC on a Payload Type match and then remove the
Payload Type table entry, but some implementations have found this leads to
problems so they have foresaken either the latching or the PT removal (or
both).

Cullen said:

"I prefer the much more explicit algorithm particularly for the RTCP
handling as implementors have a hard time figuring out wha they need to do
to process the RTCP."

[BA] I would also prefer that we have a more explicit algorithm here,
because we have seen very substantial implementation differences.  One
implementation I'm familiar with sends the RTCP packets to all RtpSender
and RtpReceiver objects.  While this is inefficient, it does avoid sending
RTCP packets to the wrong objects.  Another implementation sorts the RTCP
reports as indicated in the proposed text - but has found that the required
handling is so message-dependent that it leads to bugs (e.g. FIR and APP
messages have caused issues in particular).  So this approach is more
efficient, unless we are willing to get into detail on every RTCP message,
it will fall short.

On Fri, Dec 9, 2016 at 9:13 AM, Eric Rescorla <[email-masked]> wrote:

>> I believe we agreed that if the PT was unique, then that would be used
>> for the demux as well. I'd like to see that explicitly spelled out in this
>> text instead of just left as an option allowed but not really specified by
>> this text.  I'd also like it be clear that MID takes priority over PT.
>>
> mmusic mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmusic
>
>