Subject: [siprec] Multiplexing different participants' streams on the same port

From: "Elwell, John" <[email-masked]>
Date: Fri, 18 Mar 2011 09:11:24 +0100

Something Charles mentioned in another thread triggered me to raise this.

We consider different participants in the CS as potentially producing separate media streams that the SRC sends to the SRS, where the SRC can send them mixed or unmixed.

When sending them unmixed, up till now I think we have considered sending them to separate ports on the SRS and identifying them as separate m= lines in the SDP. Another possibility might be to multiplex the different streams on the same port and identify them by different SSRC values. I am not sure whether this would be a correct use of SSRC, and I suspect we would need to check with AVTCORE. Recent discussions in other contexts (telepresence, RTC-Web) have also drawn my attention to such possibilities. See also https://datatracker.ietf.org/doc/draft-perkins-rtcweb-rtp-usage/.

Of course, it would mean that we would have to rethink how we reference the different streams from metadata.

I am not necessarily proposing such a change - I just want to make sure we have given it due consideration.

John (as individual)

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 18 Mar 2011 08:38:20 -0400

I don't know if this is a good idea or not. But if we want to consider 
it, then we should probably enter into some discussion with CLUE, to see 
if there is any commonality.

Over in CLUE, this mixing of multiple streams over one RTP session has 
been brought up because it is a technique used in existing proprietary 
products. But the CLUE wg isn't yet to a point where it is ready to 
decide if this will be a requirement. And the AVT people have strong 
opinions on what is/isn't appropriate.

	Thanks,
	Paul

On 3/18/2011 4:11 AM, Elwell, John wrote:
> _______________________________________________
> siprec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/siprec
>

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Fri, 18 Mar 2011 08:09:43 -0700

I agree with the comment regarding CLUE, so let me take a simpler
example, and audio mixer. This mixer may mix the audio from multiple
participants.

Participant 1 (P1)
Participant 2 (P2)
Participant 3 (P3)

P1 sends audio stream with SSRC-P1
P2 sends audio stream with SSCR-P2
P2 sends audio stream with SSCR-P3

MixerA(Ma)

Ma sends mixed audio stream with SSRC SSRC-Ma and CSRC
SSRC-P1/SSRC-P2/SSRC-P3 as part of the CS.
The SRC sends this stream to the SRS for recording. I suspect it might
change the SSRC to be SSRC-SRC but maintain the CSRC information
identifying P1/P2/P3.
I expect this would be modeled as a single stream with 3 participants
P1, P2, P3. The mapping of P1 to SSRC-P1 is captured in the RTCP data
sent by SRC to SRS. In this case, there is no need to anything in the
metadata.
However, if the CSRCs are not propagated all the way to the SRS, then
perhaps participant info would need to be communicated via the metadata.

Note, I do not pretend to have this all figured out; rather, I am
putting out some ideas to see if this gels with what others have in
mind.

Cheers,
Charles

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
Behalf Of Paul Kyzivat (pkyzivat)
> Sent: Friday, March 18, 2011 5:38 AM
> To: [email-masked]
> Subject: Re: [siprec] Multiplexing different participants' streams on
the same port
> > We consider different participants in the CS as potentially
producing separate media streams that
> the SRC sends to the SRS, where the SRC can send them mixed or
unmixed.
> >
> > When sending them unmixed, up till now I think we have considered
sending them to separate ports on
> the SRS and identifying them as separate m= lines in the SDP. Another
possibility might be to
> multiplex the different streams on the same port and identify them by
different SSRC values. I am not
> sure whether this would be a correct use of SSRC, and I suspect we
would need to check with AVTCORE.
> Recent discussions in other contexts (telepresence, RTC-Web) have also
drawn my attention to such
> possibilities. See also
https://datatracker.ietf.org/doc/draft-perkins-rtcweb-rtp-usage/.
> >
> > Of course, it would mean that we would have to rethink how we
reference the different streams from
> metadata.
> >
> > I am not necessarily proposing such a change - I just want to make
sure we have given it due
> consideration.

---

From: "Parthasarathi R (partr)" <[email-masked]>
Date: Mon, 28 Mar 2011 14:12:08 +0530

Charles,

IMO, the current metadata model helps to address RTP MUX streams as
well. In terms of model, multiple participants (P1, P2, P3) metadata
block points to the single stream (S1) and stream block contains SSRC &
CSRC information. IOW, SIPREC model abstract RTP MUX stream within media
stream block.  Please let me know in case you feel that model does not
address it.

In case of metadata format, multiple "participant" elements have send
element with single stream element. The stream element has a label
attribute which points to the m-line which contains "ssrc" attribute
with CNAME parameter. I attached the example for your reference and The
example is the one of the way to represent RTP-MUX stream in the
metadata format. Please let me know the issues with this format and
example.

I agree with Paul and you that we need to understand how RTP mux stream
will be standardized but IMO, it is not the limiting factor for SIPREC
as of now.

Thanks
Partha

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Charles Eckel (eckelcu)
Sent: Friday, March 18, 2011 8:40 PM
To: Paul Kyzivat (pkyzivat); [email-masked]
Subject: Re: [siprec] Multiplexing different participants' streams on
thesame port

Participant 1 (P1)
Participant 2 (P2)
Participant 3 (P3)

MixerA(Ma)

Cheers,
Charles

the same port

unmixed.
possibility might be to

---

From: Leon Portman <[email-masked]>
Date: Tue, 29 Mar 2011 13:21:58 +0200

I like it. 

Quick question. In metadata we are using "<aor>" attribute but in SDP, it is "CNAME"? Can we use the same?

Leon

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Parthasarathi R (partr)
Sent: Monday, March 28, 2011 10:42 AM
To: Charles Eckel (eckelcu); Paul Kyzivat (pkyzivat); [email-masked]
Subject: Re: [siprec] Multiplexing different participants' streams on the same port

Charles,

IMO, the current metadata model helps to address RTP MUX streams as well. In terms of model, multiple participants (P1, P2, P3) metadata block points to the single stream (S1) and stream block contains SSRC & CSRC information. IOW, SIPREC model abstract RTP MUX stream within media stream block.  Please let me know in case you feel that model does not address it.

In case of metadata format, multiple "participant" elements have send element with single stream element. The stream element has a label attribute which points to the m-line which contains "ssrc" attribute with CNAME parameter. I attached the example for your reference and The example is the one of the way to represent RTP-MUX stream in the metadata format. Please let me know the issues with this format and example.

I agree with Paul and you that we need to understand how RTP mux stream will be standardized but IMO, it is not the limiting factor for SIPREC as of now.

Thanks
Partha

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Charles Eckel (eckelcu)
Sent: Friday, March 18, 2011 8:40 PM
To: Paul Kyzivat (pkyzivat); [email-masked]
Subject: Re: [siprec] Multiplexing different participants' streams on thesame port

I agree with the comment regarding CLUE, so let me take a simpler example, and audio mixer. This mixer may mix the audio from multiple participants.

Participant 1 (P1)
Participant 2 (P2)
Participant 3 (P3)

MixerA(Ma)

Ma sends mixed audio stream with SSRC SSRC-Ma and CSRC
SSRC-P1/SSRC-P2/SSRC-P3 as part of the CS.
The SRC sends this stream to the SRS for recording. I suspect it might change the SSRC to be SSRC-SRC but maintain the CSRC information identifying P1/P2/P3.
I expect this would be modeled as a single stream with 3 participants P1, P2, P3. The mapping of P1 to SSRC-P1 is captured in the RTCP data sent by SRC to SRS. In this case, there is no need to anything in the metadata.
However, if the CSRCs are not propagated all the way to the SRS, then perhaps participant info would need to be communicated via the metadata.

Note, I do not pretend to have this all figured out; rather, I am putting out some ideas to see if this gels with what others have in mind.

Cheers,
Charles

the same port

unmixed.
possibility might be to

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 29 Mar 2011 08:23:27 -0400

> Quick question. In metadata we are using "<aor>" attribute but in SDP, it is "CNAME"? Can we use the same?

Charles knows more than I about RTP. AFAIK CNAME is an essentially 
arbitrary string that is assigned by the media source. (Are their 
constraints on its length or format?) Perhaps it could be formatted as a 
sip AOR, perhaps not. Even if it *could*, in practice *will* it be? ISTM 
it would not be wise to count on these being the same.

We should probably allow participants to specify their cname as well as 
their AOR. This may get complicated if the same participant ends up with 
differing media streams it sends.

I also wonder if the SRC will always know the CNAMEs of the streams. I 
suspect we can't count on that.

	Thanks,
	Paul

> IMO, the current metadata model helps to address RTP MUX streams as well. In terms of model, multiple participants (P1, P2, P3) metadata block points to the single stream (S1) and stream block contains SSRC&  CSRC information. IOW, SIPREC model abstract RTP MUX stream within media stream block.  Please let me know in case you feel that model does not address it.
>

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Tue, 29 Mar 2011 07:20:38 -0700

For the format and value of the CNAME, I suggest having a look at
"Guidelines for Choosing RTP Control Protocol (RTCP) Canonical Names
(CNAMEs)", draft-ietf-avt-rtp-cnames-05. 
If the SIP AOR is meant to be the AOR registered by the participant,
this is almost certainly going to be something different.
The participant AOR is per participant. The CNAME is per media stream.

Cheers,
Charles

> From: Paul Kyzivat (pkyzivat)
> Sent: Tuesday, March 29, 2011 2:23 PM
> To: Leon Portman
> Cc: Parthasarathi R (partr); Charles Eckel (eckelcu); [email-masked]
the same port
> > Quick question. In metadata we are using "<aor>" attribute but in
SDP, it is "CNAME"? Can we use the
> same?
Behalf Of Parthasarathi R (partr)
> > IMO, the current metadata model helps to address RTP MUX streams as
well. In terms of model,
> multiple participants (P1, P2, P3) metadata block points to the single
stream (S1) and stream block
> contains SSRC&  CSRC information. IOW, SIPREC model abstract RTP MUX
stream within media stream block.
> Please let me know in case you feel that model does not address it.
> >
> > In case of metadata format, multiple "participant" elements have
send element with single stream
> element. The stream element has a label attribute which points to the
m-line which contains "ssrc"
> attribute with CNAME parameter. I attached the example for your
reference and The example is the one
> of the way to represent RTP-MUX stream in the metadata format. Please
let me know the issues with this
> format and example.
> >
> > I agree with Paul and you that we need to understand how RTP mux
stream will be standardized but
> IMO, it is not the limiting factor for SIPREC as of now.
> >
> > Thanks
> > Partha
> >
Behalf Of Charles Eckel (eckelcu)
> > I agree with the comment regarding CLUE, so let me take a simpler
example, and audio mixer. This
> mixer may mix the audio from multiple participants.
> >
> > Participant 1 (P1)
> > Participant 2 (P2)
> > Participant 3 (P3)
> >
> > The SRC sends this stream to the SRS for recording. I suspect it
might change the SSRC to be SSRC-
> SRC but maintain the CSRC information identifying P1/P2/P3.
> > I expect this would be modeled as a single stream with 3
participants P1, P2, P3. The mapping of P1
> to SSRC-P1 is captured in the RTCP data sent by SRC to SRS. In this
case, there is no need to anything
> in the metadata.
> > However, if the CSRCs are not propagated all the way to the SRS,
then perhaps participant info would
> need to be communicated via the metadata.
> >
putting out some ideas to see if
> this gels with what others have in mind.
> >
> > Cheers,
> > Charles
> >
> >> it, then we should probably enter into some discussion with CLUE,
to
> > see
> >>> Something Charles mentioned in another thread triggered me to
raise
> > this.

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 29 Mar 2011 17:40:32 -0400

Charles,

*In practice* how are cnames set for sip calls?
I'm assuming they are something canned and unrelated to the signaling 
identity.

	Thanks,
	Paul

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Tue, 29 Mar 2011 15:05:59 -0700

Typically they appear in RTCP only, and not in SIP or SDP. The RTCP
identify the mapping of CNAME to SSRC. In the SDP, you can specify the
SSRC for a media stream as described in RFC 5576 - Source-Specific Media
Attributes in the Session Description Protocol (SDP). I am not sure if
there are many existing implementations that do this.

Cheers,
Charles

> Sent: Tuesday, March 29, 2011 11:41 PM
> To: Charles Eckel (eckelcu)
> Cc: Leon Portman; Parthasarathi R (partr); [email-masked]
the same port
> >> constraints on its length or format?) Perhaps it could be formatted
as
> > a
> >> We should probably allow participants to specify their cname as
well
> > as
> >>>> Subject: Re: [siprec] Multiplexing different participants'
streams
> > on
> >>>> Over in CLUE, this mixing of multiple streams over one RTP
session
> > has
> >>>> multiplex the different streams on the same port and identify
them
> > by

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 29 Mar 2011 18:13:55 -0400

On 3/29/2011 6:05 PM, Charles Eckel (eckelcu) wrote:
> Typically they appear in RTCP only, and not in SIP or SDP.

I understand that.

I'm just wondering what is done by typical sip devices that handle both 
media and signaling - e.g. phones and gateways. Presumably *something* 
is put in for the cname. But I suspect that layering will result in the 
media layer doing its own thing without knowledge of what sort of names 
are being used in the signaling layer.

If so, just reporting the cname in the metadata might not provide any 
benefit if it can't be correlated with some signaling name.

	Thanks,
	Paul

  The RTCP

---

From: Hadriel Kaplan <[email-masked]>
Date: Wed, 30 Mar 2011 08:58:11 -0400

In practice we see all kinds of things in the RTCP cname, often unrelated to signaling info.
But you can't rely on it anyway, since you can't rely on there being any RTCP.

Furthermore, although I've (shockingly) seen some devices indicate SSRC in SDP per RFC 5576, it's rare... which means you won't know the SSRC at call setup time, so it won't be available to send in metadata at that time, requiring an update.

I humbly suggest that UDP port numbers are not in short supply, and using one per stream is both simple and effective.

-hadriel

---

From: "Elwell, John" <[email-masked]>
Date: Wed, 30 Mar 2011 16:56:20 +0200

Yes, this was the sort of discussion I was trying to provoke. Of course, SRC and SRS implementations will need to be upgraded to support the SIPREC protocol requirements, so requiring some extra things is not out of the question. Clearly one consideration might be whether middleboxes might block such features?

John (as chair)

> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Hadriel Kaplan
> Sent: 30 March 2011 13:58
> To: Paul Kyzivat
> Cc: [email-masked]; Leon Portman
> streams on the same port
> Furthermore, although I've (shockingly) seen some devices 
> indicate SSRC in SDP per RFC 5576, it's rare... which means 
> you won't know the SSRC at call setup time, so it won't be 
> available to send in metadata at that time, requiring an update.
> 
> I humbly suggest that UDP port numbers are not in short 
> supply, and using one per stream is both simple and effective.
> 
> -hadriel
> 
> 
> >>>>> Subject: Re: [siprec] Multiplexing different 
> participants' streams
> >> on
> >>>> the same port
> >>>>> 
> >>>>> 
> >>>>> 
> >>>>> sip AOR, perhaps not. Even if it *could*, in practice 
> *will* it be?
> >>>> ISTM
> >>>>> their AOR. This may get complicated if the same 
> participant ends up
> >>>> with
> >>>>>> Subject: Re: [siprec] Multiplexing different 
> participants' streams
> >>>> on the same port
> >>>>>> 
> >>>>>> Charles,
> >>>>>> 
> >>>>>> IMO, the current metadata model helps to address RTP 
> MUX streams
> >> as
> >>>> well. In terms of model,
> >>>>> contains SSRC&    CSRC information. IOW, SIPREC model 
> abstract RTP
> >> MUX
> >>>>> element. The stream element has a label attribute which 
> points to
> >> the
> >>>>>> Subject: Re: [siprec] Multiplexing different 
> participants' streams
> >>>> on thesame port
> >>>>>> 
> >>>>>>> products. But the CLUE wg isn't yet to a point where 
> it is ready
> >> to
> >>>>>>> sure whether this would be a correct use of SSRC, and 
> I suspect
> >> we
> >>>>>>> Recent discussions in other contexts (telepresence, 
> RTC-Web) have
> >>>> also

---

From: "Elwell, John" <[email-masked]>
Date: Wed, 30 Mar 2011 17:02:40 +0200

(this time as individual)

It occurs to me that if you just indicate, say, audio in the m-line and a given codec, middleboxes doing call admission control might make some assumptions about bandwidth and assume just a single stream at the rate concerned. Putting several such streams over the same path, from different participants, might cause problems. Any opinions?

John 

> participants' streams on
> [mailto:[email-masked]] On
> [mailto:[email-masked]] On
> [mailto:[email-masked]] On

---

From: "Parthasarathi R (partr)" <[email-masked]>
Date: Wed, 30 Mar 2011 22:26:07 +0530

John/Hadriel,

Please read inline.

Thanks
Partha 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Elwell, John
Sent: Wednesday, March 30, 2011 8:33 PM
To: Hadriel Kaplan; Paul Kyzivat (pkyzivat)
Cc: [email-masked]; Leon Portman
Subject: Re: [siprec] Multiplexing different participants' streams on
the same port

It occurs to me that if you just indicate, say, audio in the m-line and
a given codec, middleboxes doing call admission control might make some
assumptions about bandwidth and assume just a single stream at the rate
concerned. 
<partha> IMO, This problem will be solved in case bandwidth attribute
under the single stream is populated properly. If it is the problem,
then the problem is outside the scope of SIPREC. </partha>

Putting several such streams over the same path, from different
participants, might cause problems. Any opinions?

John 

> Furthermore, although I've (shockingly) seen some devices indicate 
> SSRC in SDP per RFC 5576, it's rare...
<partha> May take time to adopt..As less RTP MUX in single stream
implementation may be one of the reason. </partha>
 which means you won't know the 
> SSRC at call setup time, so it won't be available to send in metadata 
> at that time, requiring an update.
> 
<partha> I guess that it is possible to populate SSRC while offer is
sent out and it is implementation specific </partha>
  
> I humbly suggest that UDP port numbers are not in short supply, and 
> using one per stream is both simple and effective.
<partha> may not always the case. There are nightmare with multiple
m-lines passing through some middle boxes </partha?>

---

From: Hadriel Kaplan <[email-masked]>
Date: Wed, 30 Mar 2011 16:03:19 -0400

Maybe I misunderstood the context of this thread.  The email thread appeared to be about using the SSRCs or cnames of the real participant streams, not new ones of the session recording client.  In other words, in RTP RFC 3550 lingo the recording client would be doing a RTP translator model vs. an RTP mixer or endpoint model.  But based on your comments below I take it  you're talking about the recording client being a mixer or endpoint model instead?  If so, then sure you could make the recording client do whatever you want. (and pay the price penalty for doing so)

-hadriel

---

From: "Henry Lum" <[email-masked]>
Date: Wed, 30 Mar 2011 13:29:39 -0700

One of the downside I see with multiplexing of streams is not being able
to pause/resume a recorded stream for a specific participant. Our
current proposed mechanism is to reINVITE with a=inactive but this
pauses all the participant streams in the m= line. We could come up with
another mechanism to pause/resume at a finer granularity, but then we
could end up with multiple mechanisms for this feature.

Henry

> Behalf Of Hadriel Kaplan
> Sent: Wednesday, March 30, 2011 4:03 PM
> To: Elwell, John
> Maybe I misunderstood the context of this thread.  The email thread
> appeared to be about using the SSRCs or cnames of the real participant
> streams, not new ones of the session recording client.  In other
words,
> in RTP RFC 3550 lingo the recording client would be doing a RTP
> translator model vs. an RTP mixer or endpoint model.  But based on
your
> comments below I take it  you're talking about the recording client
> being a mixer or endpoint model instead?  If so, then sure you could
> make the recording client do whatever you want. (and pay the price
> penalty for doing so)
> 
> -hadriel
> 
> 
> > Yes, this was the sort of discussion I was trying to provoke. Of
> course, SRC and SRS implementations will need to be upgraded to
support
> the SIPREC protocol requirements, so requiring some extra things is
not
> out of the question. Clearly one consideration might be whether
> middleboxes might block such features?
> >
> > John (as chair)
> >
> >

-------------------------------------------------------------------------------------------------------------------
CONFIDENTIALITY NOTICE: This e-mail and any files attached may contain confidential and proprietary information of Alcatel-Lucent and/or its affiliated entities. Access by the intended recipient only is authorized. Any liability arising from any party acting, or refraining from acting, on any information contained in this e-mail is hereby excluded. If you are not the intended recipient, please notify the sender immediately, destroy the original transmission and its attachments and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Copyright in this e-mail and any attachments belongs to Alcatel-Lucent and/or its affiliated entities.

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 31 Mar 2011 03:48:04 -0400

The way I took this is:

- the SRC could combine multiple CS streams into a single
   RS m-line, while distinguishing the streams by CNAME.

- the SRC could be receiving a "mixed" stream from CS
   and have info about the CNAMEs and their relationship
   to participans that it wants to pass on

- the SRC could be receiving a stream with multiple
   CNAMEs but not know anything about them, and so describe
   the participant(s) as it knows them without regard to
   CNAMEs.

- the SRC could be passing every CS stream as a separate
   m-line in the RS, with one participant for each.
   Probably each will have a single CNAME but this will not
   be mentioned.

	Thanks,
	Paul