Subject: [AVTCORE] BUNDLE: a single stream with multiple MIDs?

From: Jonathan Lennox <[email-masked]>
Date: Thu, 7 Jul 2016 22:36:40 +0000

Hi, all —

(This was an issue I raised in AVTCORE in Buenos Aires — I promised to send e-mail to the list but hadn’t remembered to get around to it until now.)

When finishing up the CLUE RTP mapping draft, I realized that one of CLUE’s RTP requirements didn’t actually end up getting satisfied by BUNDLE (which is the solution CLUE converged on).  This isn’t a CLUE-specific issue, so I’m raising it here.

The issue is whether we want to support a use case, in BUNDLE, where a single RTP stream corresponds to more than media description (and thus has more than one MID value)?

The use case is where one m-line has a semantic of “always view this person” (my boss, say, or my customer); and another m-line has a semantic of “the current loudest speaker”. (In CLUE, these would be a single content capture in the former case, and a switched capture for the latter.)  Whenever the boss *is* the current loudest speaker, the same content would be sent for both m-lines.

A naive implementation would simply duplicate all the packets for the two m-lines, with different MID values, but this has two problems.  First off all, it obviously wastes bandwidth.  Potentially more seriously, it precludes any RTP middlebox topology which doesn’t rewrite SSRC values, since the same content (arriving with a single SSRC value at the middlebox) would need to be sent with two different SSRC values down from the middlebox.  If PERC goes with its current consensus of no SSRC rewriting, this will particularly be a problem for PERC.

I certainly don’t think this is something that should block BUNDLE’s completion, but I think it could be a pretty easy extension.  

(My initial design proposal was to allow multiple SDES values of the same type, and multiple SDES header extensions items of the same type, to be sent simultaneously in RTP — protocol-syntactically this is trivial, you’d just need to negotiate that you support it.  But I’m not wedded to this solution.)

What do people think — is this worth working on?  Is there interest in discussing it in Berlin, and if so, in what venue?

---

From: Suhas Nandakumar <[email-masked]>
Date: Fri, 8 Jul 2016 06:22:01 +0530

Hi Jonathan

  Clarification question :

   if we have 2 m=lines

m = ...
a=mid1
<manager>

m=...
a=mid2
<loudest speaker>

does these 2 correspond to same media source ? It felt so from the
duplicate design you mentioned but wanted to confirm

Thanks
Suhas

On Fri, Jul 8, 2016 at 4:06 AM, Jonathan Lennox <[email-masked]> wrote:

> Hi, all —
>
> (This was an issue I raised in AVTCORE in Buenos Aires — I promised to
> send e-mail to the list but hadn’t remembered to get around to it until
> now.)
>
> When finishing up the CLUE RTP mapping draft, I realized that one of
> CLUE’s RTP requirements didn’t actually end up getting satisfied by BUNDLE
> (which is the solution CLUE converged on).  This isn’t a CLUE-specific
> issue, so I’m raising it here.
>
> The issue is whether we want to support a use case, in BUNDLE, where a
> single RTP stream corresponds to more than media description (and thus has
> more than one MID value)?
>
> The use case is where one m-line has a semantic of “always view this
> person” (my boss, say, or my customer); and another m-line has a semantic
> of “the current loudest speaker”. (In CLUE, these would be a single content
> capture in the former case, and a switched capture for the latter.)
> Whenever the boss *is* the current loudest speaker, the same content would
> be sent for both m-lines.
>
> A naive implementation would simply duplicate all the packets for the two
> m-lines, with different MID values, but this has two problems.  First off
> all, it obviously wastes bandwidth.  Potentially more seriously, it
> precludes any RTP middlebox topology which doesn’t rewrite SSRC values,
> since the same content (arriving with a single SSRC value at the middlebox)
> would need to be sent with two different SSRC values down from the
> middlebox.  If PERC goes with its current consensus of no SSRC rewriting,
> this will particularly be a problem for PERC.
>
> I certainly don’t think this is something that should block BUNDLE’s
> completion, but I think it could be a pretty easy extension.
>
> (My initial design proposal was to allow multiple SDES values of the same
> type, and multiple SDES header extensions items of the same type, to be
> sent simultaneously in RTP — protocol-syntactically this is trivial, you’d
> just need to negotiate that you support it.  But I’m not wedded to this
> solution.)
>
> What do people think — is this worth working on?  Is there interest in
> discussing it in Berlin, and if so, in what venue?
> _______________________________________________
> mmusic mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmusic
>

---

From: Jonathan Lennox <[email-masked]>
Date: Fri, 8 Jul 2016 15:51:28 +0000

The intention of my proposal is that these could indeed be (temporarily) the same RTP media source, even though they’re two separate m-lines.

On Jul 7, 2016, at 8:52 PM, Suhas Nandakumar <[email-masked]<mailto:[email-masked]>> wrote:

Hi Jonathan

  Clarification question :

   if we have 2 m=lines

m = ...
a=mid1
<manager>

m=...
a=mid2
<loudest speaker>

does these 2 correspond to same media source ? It felt so from the duplicate design you mentioned but wanted to confirm

Thanks
Suhas

On Fri, Jul 8, 2016 at 4:06 AM, Jonathan Lennox <[email-masked]<mailto:[email-masked]>> wrote:
Hi, all —

What do people think — is this worth working on?  Is there interest in discussing it in Berlin, and if so, in what venue?
_______________________________________________
mmusic mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/mmusic

---

From: Peter Thatcher <[email-masked]>
Date: Mon, 11 Jul 2016 10:42:44 -0700

Basically what your're asking for is header extension that says "Not only
am I MID X, but I'm also, temporarily, MID Y"?   Why not just have a
second, separate, header extension for "I'm also temporarily MID Y"?   Then
the existing semantics of BUNDLE and MID (and RIDs, which are scoped to
MIDs) need not change, but you get the information you need (I think).

On Fri, Jul 8, 2016 at 8:51 AM, Jonathan Lennox <[email-masked]> wrote:

> The intention of my proposal is that these could indeed be (temporarily)
> the same RTP media source, even though they’re two separate m-lines.
>
> On Jul 7, 2016, at 8:52 PM, Suhas Nandakumar <[email-masked]> wrote:
>
> Hi Jonathan
>
>   Clarification question :
>
>    if we have 2 m=lines
>
> m = ...
> a=mid1
> <manager>
>
> m=...
> a=mid2
> <loudest speaker>
>

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Tue, 12 Jul 2016 18:27:08 +0200

2016-07-11 19:42 GMT+02:00 Peter Thatcher <[email-masked]>:
> Basically what your're asking for is header extension that says "Not only am
> I MID X, but I'm also, temporarily, MID Y"?   Why not just have a second,
> separate, header extension for "I'm also temporarily MID Y"?   Then the
> existing semantics of BUNDLE and MID (and RIDs, which are scoped to MIDs)
> need not change, but you get the information you need (I think).

Agreed.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Jonathan Lennox <[email-masked]>
Date: Tue, 12 Jul 2016 16:42:17 +0000

On Jul 11, 2016, at 1:42 PM, Peter Thatcher <[email-masked]<mailto:[email-masked]>> wrote:

Basically what your're asking for is header extension that says "Not only am I MID X, but I'm also, temporarily, MID Y"?   Why not just have a second, separate, header extension for "I'm also temporarily MID Y"?   Then the existing semantics of BUNDLE and MID (and RIDs, which are scoped to MIDs) need not change, but you get the information you need (I think).

The thing is, all MID values are, conceptually, temporary — they can change at any time.  So is it really meaningful to distinguish short-term temporary from long-term temporary?  Where do you draw the line?

On Fri, Jul 8, 2016 at 8:51 AM, Jonathan Lennox <[email-masked]<mailto:[email-masked]>> wrote:
The intention of my proposal is that these could indeed be (temporarily) the same RTP media source, even though they’re two separate m-lines.

Hi Jonathan

  Clarification question :

   if we have 2 m=lines

m = ...
a=mid1
<manager>

m=...
a=mid2
<loudest speaker>

Thanks
Suhas

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 13 Jul 2016 07:20:23 +0000

Hi,

A few questions:

Q1:

If an RTP packet is associated with two or more m- lines, which codecs payload type values, SSRC values should it use? Can it pick from any of the associated m- lines, or does one of the MIDs have to be the “primary”, while the others are “secondary”, “temporary”, or whatever we want to call them…

Q2:

Currently, draft-bundle says:

   "The RTP MID header extension SHOULD be included in some RTP packets
   at the start of the session and whenever the SSRC changes."

Now, when the “loudest speaker” changes, the SSRC does not change (at least not the SSRC of the “boss”). But, there must still be a “I am no longer also MID Y” indication.

At the same time, if the “loudest speaker” is now associated with another m- line (e.g. “secretary”), there must not be an indication that the “secretary” MID and “loudest speaker” MID are associated.

Regards,

Christer

From: mmusic <[email-masked]<mailto:[email-masked]>> on behalf of Jonathan Lennox <[email-masked]<mailto:[email-masked]>>
Date: Tuesday 12 July 2016 at 19:42
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [MMUSIC] BUNDLE: a single stream with multiple MIDs?

Hi Jonathan

  Clarification question :

   if we have 2 m=lines

m = ...
a=mid1
<manager>

m=...
a=mid2
<loudest speaker>

Thanks
Suhas

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Wed, 13 Jul 2016 10:53:55 +0200

2016-07-08 0:36 GMT+02:00 Jonathan Lennox <[email-masked]>:
> The use case is where one m-line has a semantic of “always view this person” (my boss, say, or my customer); and another m-line has a semantic of “the current loudest speaker”.

Why is another m-line required for the "current loudest speaker" at all?

Who the current speaker is can be known by means of RTP header
extensions in client side. It can be also signaled by the server in
any custom way (out of the scope of SDP). Any of those mechanisms is
better than having yet another artificial m-line for that purpose.

I don't know CLUE, but assuming it pursuits similar aim than BUNDLE, I
think we should keep to a more declarative SDP syntax in which each
m-line represents a specific "media stream track" from a single
participant in the session, identified by a fixed MID (plus RID if the
sender switches to another media encoding).

If we can not even guarantee that, then SDP becomes extremely hard to
understand.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 13 Jul 2016 09:27:35 +0000

Hi,

CLUE and BUNDLE are two separate things, and CLUE doesn¹t require usage of
BUNDLE.

Regards,

Christer

On 13/07/16 11:53, "avt on behalf of Iñaki Baz Castillo"
<[email-masked] on behalf of [email-masked]> wrote:

>> The use case is where one m-line has a semantic of ³always view this
>>person² (my boss, say, or my customer); and another m-line has a
>>semantic of ³the current loudest speaker².
>
>Audio/Video Transport Core Maintenance
>[email-masked]
>https://www.ietf.org/mailman/listinfo/avt

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 13 Jul 2016 11:29:20 +0200

Den 2016-07-08 kl. 00:36, skrev Jonathan Lennox:

> (My initial design proposal was to allow multiple SDES values of the
> same type, and multiple SDES header extensions items of the same
> type, to be sent simultaneously in RTP — protocol-syntactically this
> is trivial, you’d just need to negotiate that you support it.  But
> I’m not wedded to this solution.)
>

So, if one is going to enable multiple concurrent values for SDES items 
that clearly needs an RTP extension which defines how one does. It also 
needs to discuss which SDES items of the existing that can be used in 
this way and what it means. There also needs to be clear procedures for 
how one deals with updates of the current set. Clearly this needs to be 
a signalled extension as you say.

I think this is clearly work belonging to the AVTCORE WG.

> What do people think — is this worth working on?  Is there interest
> in discussing it in Berlin, and if so, in what venue?

I think this is worth some more thinking. My review of the RID drafts 
mad it clear to me that we might not have sufficiently thought through 
how the scoping and interactions are between the different new 
labellings we have for RTP stream. And this also points at another 
aspect, the expectations of what is produced from a media description. 
For media source that are what I call conceptual, like the "current 
speaker" or role based like "meeting chair", there clearly need to be 
understanding how this interacts in the signalling solutions.

I am sorry to say that I have lost track of where CLUE has ended up and 
the interaction between SDP and the CLUE layer.

I don't want to commit to anything, but I am happy to continue 
discussing this.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Services, Media and Network features, Ericsson Research EAB/TXM
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Färögatan 6                 | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Wed, 13 Jul 2016 11:47:32 +0200

> CLUE and BUNDLE are two separate things, and CLUE doesn¹t require usage of

Fine with me. However, my main question remains valid:

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 13 Jul 2016 09:52:45 +0000

Hi,

>Why is another m-line required for the "current loudest speaker" at all?

Assume you have a conference client with two screens: one constantly shows
the “boss”, and one shows the “loudest speaker”.

You have one m- line to describe the “boss” RTP stream.

But, you also need an m- line to describe the “loudest speaker” RTP
stream, because often it will NOT be the same as the “boss” RTP stream.

Regards,

Christer

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Wed, 13 Jul 2016 11:56:47 +0200

> stream, because often it will NOT be the same as the “boss” RTP stream.

Why should a remote video representation be tiled to a specific m-line?

How media streams/tracks are represented in the SDP and how the
frontend app renders them are different things.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 13 Jul 2016 10:10:43 +0000

Hi,

>> the ³boss², and one shows the ³loudest speaker².
>>
>> You have one m- line to describe the ³boss² RTP stream.
>>
>> But, you also need an m- line to describe the ³loudest speaker² RTP
>> stream, because often it will NOT be the same as the ³boss² RTP stream.
>

SDP is not used to describe how the streams are rendered. The CLUE
protocol is used to define that.

>From an SDP perspective, there are two RTP streams, so there are two m-
lines.

Regards,

Christer

---

From: Jonathan Lennox <[email-masked]>
Date: Wed, 13 Jul 2016 14:14:28 +0000

On Jul 13, 2016, at 3:20 AM, Christer Holmberg <[email-masked]<mailto:[email-masked]>> wrote:

Hi,

A few questions:

Q1:

It would have to satisfy all of them — in situations where that's not possible (e.g. if the two m-lines specify non-overlapping codec sets) this couldn’t be used.

a=ssrc would probably not be usable in this model, but it’s probably in general not usable for a loudest-speaker-switching m-line which is switching SSRC (as opposed to CSRC).

Q2:

I think it’s clearer if you interpret “when the SSRC changes” as if it were “whenever the mapping between SSRC and MID changes”.  This means the same thing if there’s only one MID per SSRC, but if you have multiple MID values for an SSRC you have to re-send the whole (new) set whenever the set changes.

Regards,

Christer

Hi Jonathan

  Clarification question :

   if we have 2 m=lines

m = ...
a=mid1
<manager>

m=...
a=mid2
<loudest speaker>

Thanks
Suhas

---

From: Suhas Nandakumar <[email-masked]>
Date: Wed, 13 Jul 2016 20:20:17 +0530

> just need to negotiate that you support it.  But I’m not wedded to this
Thinking a bit more on this .. it seems like a nice way forward . I agree
with Magnus that it needs a way for the recipient to understand
addition/removal of an item from the set ..
If the set represent duplicates , then when the boss is the loud speaker,
the set will contain 2 mids .. when the loudest speaker is not the boss,
the set will have just one item in this example and the receiver will do
the normal processing of the loudest speaker .. From the mux point of view
these MIDs must be IDENTICAL-PER-PT

---

From: Jonathan Lennox <[email-masked]>
Date: Wed, 13 Jul 2016 14:58:50 +0000

On Jul 13, 2016, at 10:50 AM, Suhas Nandakumar <[email-masked]<mailto:[email-masked]>> wrote:

Thinking a bit more on this .. it seems like a nice way forward . I agree with Magnus that it needs a way for the recipient to understand addition/removal of an item from the set ..

My thinking was that you always send the complete set, if you send any MID at all. Thus, when you go from 2 MIDs to 1 MID, you send 1 MID, and the receiver knows that the other MID no longer applies.  (This rule applies both for header extensions and SDES items.)

If the set represent duplicates , then when the boss is the loud speaker, the set will contain 2 mids .. when the loudest speaker is not the boss, the set will have just one item in this example and the receiver will do the normal processing of the loudest speaker .. From the mux point of view these MIDs must be IDENTICAL-PER-PT

Right.

---

From: Suhas Nandakumar <[email-masked]>
Date: Wed, 13 Jul 2016 20:32:20 +0530

On Wed, Jul 13, 2016 at 8:28 PM, Jonathan Lennox <[email-masked]> wrote:

>
> On Jul 13, 2016, at 10:50 AM, Suhas Nandakumar <[email-masked]>
> wrote:
>
>
>
> My thinking was that you always send the complete set, if you send any MID
> at all. Thus, when you go from 2 MIDs to 1 MID, you send 1 MID, and the
> receiver knows that the other MID no longer applies.  (This rule applies
> both for header extensions and SDES items.)
>

Agreed..  That's the point i was probably trying to make in my example
earlier. There might be few explanations needed on handling as you implied.

---

From: Peter Thatcher <[email-masked]>
Date: Wed, 13 Jul 2016 11:14:35 -0700

On Tue, Jul 12, 2016 at 9:42 AM, Jonathan Lennox <[email-masked]> wrote:

> The thing is, all MID values are, conceptually, temporary — they can
> change at any time.  So is it really meaningful to distinguish short-term
> temporary from long-term temporary?  Where do you draw the line?
>

​That doesn't match my mental model of MIDs.   I've always thought of an
RTP stream as being bound to a particular MID, not able to change between
various MIDs.  I might be wrong, but it seems like there are weird edge
cases with switching MIDs, such as how RIDs play into it (since they are
scoped to MIDs).  Another is the demux algorithm in WebRTC/ORTC which
"locks on" to SSRCs and only looks at MIDs if the SSRCs are unknown (the
demux algorithm certainly does not take into account demuxing a packet into
multiple RtpReceivers).

So it sounds like you are asking for two things:
1.  Be able to switch MIDs
2.  Be able to have > 1 MIDs.

And both of those blow up my mental model of MIDs and could cause problems
with other things.

I think it would be less risky to define a separate header extension for
"multiplicity of changable MIDs" rather than re-using the existing header
extension, when the semantics seems so much different (than my mental
model, at least).

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 14 Jul 2016 09:03:16 +0000

Hi,

First, would this really be worth the effort?

It’s true that sending two identical streams (in Jonathan’s case where “boss” and “loudest speaker” are the same persons) is a resource waste. But, the needed resources must be available anyway, as the “loudest speaker” can change at any time.

Also, there seems to be so many constraints (the codecs, payload types etc must overlap), so the question is whether it would be usable to begin with.

Second, IF we want to do something like this, should it instead be done on a higher application/CLUE layer?  Why start mixing up the lower-layer MID semantics etc?

Regards,

Christer

From: avt <[email-masked]<mailto:[email-masked]>> on behalf of "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Date: Wednesday 13 July 2016 at 21:14
To: Jonathan Lennox <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [AVTCORE] [MMUSIC] BUNDLE: a single stream with multiple MIDs?

On Tue, Jul 12, 2016 at 9:42 AM, Jonathan Lennox <[email-masked]<mailto:[email-masked]>> wrote:

On Jul 11, 2016, at 1:42 PM, Peter Thatcher <[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>> wrote:

​That doesn't match my mental model of MIDs.   I've always thought of an RTP stream as being bound to a particular MID, not able to change between various MIDs.  I might be wrong, but it seems like there are weird edge cases with switching MIDs, such as how RIDs play into it (since they are scoped to MIDs).  Another is the demux algorithm in WebRTC/ORTC which "locks on" to SSRCs and only looks at MIDs if the SSRCs are unknown (the demux algorithm certainly does not take into account demuxing a packet into multiple RtpReceivers).

And both of those blow up my mental model of MIDs and could cause problems with other things.

I think it would be less risky to define a separate header extension for "multiplicity of changable MIDs" rather than re-using the existing header extension, when the semantics seems so much different (than my mental model, at least).

On Fri, Jul 8, 2016 at 8:51 AM, Jonathan Lennox <[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>> wrote:
The intention of my proposal is that these could indeed be (temporarily) the same RTP media source, even though they’re two separate m-lines.

On Jul 7, 2016, at 8:52 PM, Suhas Nandakumar <[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>> wrote:

Hi Jonathan

  Clarification question :

   if we have 2 m=lines

m = ...
a=mid1
<manager>

m=...
a=mid2
<loudest speaker>

Thanks
Suhas

On Fri, Jul 8, 2016 at 4:06 AM, Jonathan Lennox <[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>> wrote:
Hi, all —

What do people think — is this worth working on?  Is there interest in discussing it in Berlin, and if so, in what venue?
_______________________________________________
mmusic mailing list
[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>
https://www.ietf.org/mailman/listinfo/mmusic

_______________________________________________
mmusic mailing list
[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>
https://www.ietf.org/mailman/listinfo/mmusic

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Thu, 14 Jul 2016 11:13:45 +0200

2016-07-14 11:03 GMT+02:00 Christer Holmberg <[email-masked]>:
> Second, IF we want to do something like this, should it instead be done on a
> higher application/CLUE layer?  Why start mixing up the lower-layer MID
> semantics etc?

Some ugly stuff regarding SDP is the fact that m-lines are indexed by
their position, which means that they cannot be removed within the
same session.

Then a=mid comes to the rescue and provides a per m-line unique
identifier, leaving the door open for a future spec change allowing
m-lines removal (have you ever seen how an SDP from a SFU with
participants entering and leaving the room grows?).

The current proposal breaks that. Please let's MID to be a single,
unique and mandatory attribute per m-line.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 14 Jul 2016 09:57:38 +0000

>...

Šand let each RTP packet be associated with one, and only one, of those
MIDs/m-lines.

You can then have application logic saying e.g. ³If no Œloudest-speaker¹
media is received, then duplicate the Œboss¹ media on the screen
associated with the loudest speaker². It¹s much more flexible, and without
any of the constraints mentioned earlier.

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 14 Jul 2016 10:00:21 +0000

…and in addition, doing it as application logic also allows you to do it
with non-BUNDLE.

On 14/07/16 12:57, "mmusic on behalf of Christer Holmberg"
<[email-masked] on behalf of [email-masked]>

---

From: Miguel París Díaz <[email-masked]>
Date: Thu, 14 Jul 2016 12:19:51 +0200

Hello everyone,
firstable, thanks Jonathan to open this interesting discussion.

For me it is quite important not only having real time
applications/infrastructures working, but also in an efficient way, because
it could be the different between success or failure...
Why? Because the usage of technology is subject of business, where the
costs are essential.

Taking this into account and that in the real world:
  - For conferences with 3 or less participants (usually) the best approach
is doing p2p.
  - Conferences with 4 or more participants (usually) use a midlebox like
an SFU, etc.
  - Most conferences have less than 5/6 participants

In the Jonathan's case with a SFU:
  - Sending all streams, there would be nParticipants input streams and
nParticipants * nParticipants output streams. Total streams: (nParticipants
+ 1) * nParticipants
  - With his proposal, there would be nParticipants input streams and
(nParticipants -1) * nParticipants output streams. Total streams:
nParticipants * nParticipants, so we save nParticipants streams.

And the percentage of streams saved is: 1 - ( (nParticipants *
nParticipants) / (nParticipants + 1) * nParticipants)) = 1 - (nParticipants
/ (nParticipants + 1))

Assuming that in an SFU the cost of receiving and sending streams is almost
the same, with this approach we could save CPU and reduce the bandwidth
usage with conferences of:
  - 4 participants: 1 -4/5 = 1-0.8 = 20 %
  - 5 participatns = 1- 5/6 = 1 - 0.833 = 16.6 %
  - 6 participants = 1 - 6/7 = 1 - 0.857 = 14.3 %

We also should consider that using simulcast of SVC for video would reduce
this percentages.
But in conclusion, let's take into account the efficiency, not only for
that but in general please ;)

Related to that:

> participants entering and leaving the room grows?).

To avoid huge SDP, there is draft proposing partial offers and answers:
https://tools.ietf.org/html/draft-roach-mmusic-pof-pan

Best regards!!

-- 
Miguel París Díaz
------------------------------------------------------------------------
Computer/Software engineer.
Researcher and architect in http://www.kurento.org
http://twitter.com/mparisdiaz
------------------------------------------------------------------------

---

From: Jonathan Lennox <[email-masked]>
Date: Thu, 14 Jul 2016 14:23:04 +0000

> On Jul 14, 2016, at 5:13 AM, Iñaki Baz Castillo <[email-masked]> wrote:
> 

I’m not proposing to change this.  There will still be a single MID per m-line.

What I’m proposing instead is that the mapping between RTP Streams and m-lines is not necessarily 1-to-1 — an RTP stream can provide the media for more than one m-line at a time.

---

From: Suhas Nandakumar <[email-masked]>
Date: Thu, 14 Jul 2016 21:13:25 +0530

> any of the constraints mentioned earlier.
I agree it brings in flexibility and it scares me a bit as well .. Each
application needs to define their own logic or need a common logic across
all applications ..

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 14 Jul 2016 15:49:18 +0000

Hi,

You don't need a common logic. You can define an application layer protocol (e.g. CLUE) to define/configure/negotiate these kind of things.

Regards,

Christer

Sent from my Windows Phone
________________________________
From: Suhas Nandakumar<mailto:[email-masked]>
Sent: ‎14/‎07/‎2016 18:43
To: Christer Holmberg<mailto:[email-masked]>
Cc: Iñaki Baz Castillo<mailto:[email-masked]>; Jonathan Lennox<mailto:[email-masked]>; mmusic<mailto:[email-masked]>; IETF AVTCore WG<mailto:[email-masked]>
Subject: Re: [MMUSIC] [AVTCORE] BUNDLE: a single stream with multiple MIDs?

I agree it brings in flexibility and it scares me a bit as well .. Each application needs to define their own logic or need a common logic across all applications ..

Regards,

Christer

---

From: Miguel París Díaz <[email-masked]>
Date: Wed, 20 Jul 2016 13:36:59 +0200

Hello,
thinking a bit about that idea, if we support that different RTP streams
have the same MID, we should provide a kind of version to deal with race
conditions.

Taking the example of “the current loudest speaker” (LS) , if the middle
box switches many times Alice <-> Bob streams the endpoint receiving the
RTP streams may show the incorrect loudest-speaker due to processing an RTP
stream before the other one (network, race condition into the endpoint,
etc.)

Imagine that case:
MidleBox side: Alice LS -> Bob LS -> Alice LS
Endpoint side: Process Alice as LS -> Process Alice LS -> Process Bob LS

At the end the endpoint would render Bob as LS, but it is wrong.

To fix this race condition problem I propose adding a version:
MidleBox side: Alice LS-0 -> Bob LS-1 -> Alice LS-2
Endpoint side: Process Alice as LS-0 -> Process Alice LS-2 -> Process Bob
LS-1 (discarded because 1<2)

What do you think about that?

Best regards!!

2016-07-14 17:49 GMT+02:00 Christer Holmberg <[email-masked]
>:

> Hi,
>
> You don't need a common logic. You can define an application layer
> protocol (e.g. CLUE) to define/configure/negotiate these kind of things.
>
> Regards,
>
> Christer
>
> ------------------------------
> From: Suhas Nandakumar <[email-masked]>
> To: Christer Holmberg <[email-masked]>
> Cc: Iñaki Baz Castillo <[email-masked]>; Jonathan Lennox
> <[email-masked]>; mmusic <[email-masked]>; IETF AVTCore WG
> <[email-masked]>

---

From: Jonathan Lennox <[email-masked]>
Date: Wed, 20 Jul 2016 15:46:45 +0000

draft-ietf-avtext-sdes-hdr-ext discusses ways to notice reordering of packets carrying SDES header extensions (of which MID is an instance).  Is that text sufficient?

On Jul 20, 2016, at 1:36 PM, Miguel París Díaz <[email-masked]<mailto:[email-masked]>> wrote:

Hello,
thinking a bit about that idea, if we support that different RTP streams have the same MID, we should provide a kind of version to deal with race conditions.

Taking the example of “the current loudest speaker” (LS) , if the middle box switches many times Alice <-> Bob streams the endpoint receiving the RTP streams may show the incorrect loudest-speaker due to processing an RTP stream before the other one (network, race condition into the endpoint, etc.)

Imagine that case:

To fix this race condition problem I propose adding a version:
MidleBox side: Alice LS-0 -> Bob LS-1 -> Alice LS-2
Endpoint side: Process Alice as LS-0 -> Process Alice LS-2 -> Process Bob LS-1 (discarded because 1<2)

Best regards!!

2016-07-14 17:49 GMT+02:00 Christer Holmberg <[email-masked]<mailto:[email-masked]>>:
Hi,

Regards,

Christer

Regards,

Christer

--
Miguel París Díaz
------------------------------------------------------------------------
Computer/Software engineer.
Researcher and architect in http://www.kurento.org<http://www.kurento.org/>
http://twitter.com/mparisdiaz
------------------------------------------------------------------------

---

From: Miguel París Díaz <[email-masked]>
Date: Fri, 22 Jul 2016 09:05:29 +0200

Hello Jonathan,
Do you mean to add MID SDES Item [1] using SDES header extensions [2] and
also extends that to provide the "version"?
If we use this approach, RTP MID Header Extension [3] would be useless,
wouldn't it?

Anyway, could you expand your idea to show how would you solve the race
condition problem?

Best!!

Refs
[1]
https://tools.ietf.org/html/draft-ietf-mmusic-sdp-bundle-negotiation-31#section-14.2
[2] https://tools.ietf.org/html/draft-ietf-avtext-sdes-hdr-ext-07
[3]
https://tools.ietf.org/html/draft-ietf-mmusic-sdp-bundle-negotiation-31#section-14.3

2016-07-20 17:46 GMT+02:00 Jonathan Lennox <[email-masked]>:

> draft-ietf-avtext-sdes-hdr-ext discusses ways to notice reordering of
> packets carrying SDES header extensions (of which MID is an instance).  Is
> that text sufficient?
>
> On Jul 20, 2016, at 1:36 PM, Miguel París Díaz <[email-masked]>
> wrote:
>
> Hello,
> 2016-07-14 17:49 GMT+02:00 Christer Holmberg <
> [email-masked]>:
>
>> Hi,
>>

---

From: Jonathan Lennox <[email-masked]>
Date: Mon, 25 Jul 2016 18:16:39 +0000

On Jul 22, 2016, at 3:05 AM, Miguel París Díaz <[email-masked]<mailto:[email-masked]>> wrote:

Hello Jonathan,
Do you mean to add MID SDES Item [1] using SDES header extensions [2] and also extends that to provide the "version"?
If we use this approach, RTP MID Header Extension [3] would be useless, wouldn't it?

The plan is to update the definition of the MID Header Extension in BUNDLE to use the SDES header extensions draft.  This isn’t a change to the bits on the wire or in the signaling, just in how it’s specified.

Anyway, could you expand your idea to show how would you solve the race condition problem?

Since SDES is sent periodically, even if packet reordering causes endpoint confusion (which is unlikely, since MID is only supposed to be sent “early” in a stream), the endpoint should resynchronize pretty quickly.

Best!!

Refs
[1] https://tools.ietf.org/html/draft-ietf-mmusic-sdp-bundle-negotiation-31#section-14.2
[2] https://tools.ietf.org/html/draft-ietf-avtext-sdes-hdr-ext-07
[3] https://tools.ietf.org/html/draft-ietf-mmusic-sdp-bundle-negotiation-31#section-14.3

2016-07-20 17:46 GMT+02:00 Jonathan Lennox <[email-masked]<mailto:[email-masked]>>:
draft-ietf-avtext-sdes-hdr-ext discusses ways to notice reordering of packets carrying SDES header extensions (of which MID is an instance).  Is that text sufficient?

Hello,

Imagine that case:

Best regards!!

Regards,

Christer

Regards,

Christer