Subject: [MMUSIC] How does mmt grouping work with multiple streams of the same type?

From: "Stach, Thomas" <[email-masked]>
Date: Thu, 11 Oct 2012 13:12:52 +0000

Christer, Harald, Jonathan,

I have the impression that the draft-holmberg-mmusic-sdp-mmt-negotiation is based on Jonathan's proposal 
to have an explicit desrciption for the bundled streams as proposed in
http://www.ietf.org/mail-archive/web/mmusic/current/msg09454.html

Among other points the following is stated:
* The m=bundle is a complete description of an entirely separate m-line; if it's used, the other m-lines in the group are rejected and the information they carry is ignored.
I think this a desirable property

Now, consider a case were you have 4 streams, i.e. 4 m-lines (e.g. 2 audio and 2 video).
How would that be mapped to the m=anymedia line? 
How does mmt grouping work with multiple streams of the same type?

>From the currently proposed syntax for the m=anymedia line I cannot deduce that multiple audio or video streams are offered.
I would have to look into the "legacy" m-lines to recognize this.
In case that e.g. the video stream use different codecs, I could also learn that only from the legacy m-lines.

Regards
Thomas

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 11 Oct 2012 15:40:32 +0200

> In case that e.g. the video stream use different codecs, I could also learn that only from the legacy m-lines.

In the case where it's desirable to carry multiple video streams in one 
RTP session, and have different descriptions in SDP for the different 
video streams, there has to be some signalling that tells the recipient 
which description applies to which video stream.

The video streams will have different SSRCSs. Therefore, the only 
reasonable answer in the MMT grouping case seems to be SSRC-associated 
signalling.

In the other variants, it's actually unclear to me how the video streams 
described by the different m= lines would be detected by the recipients; 
I'd been assuming SSRC-associated signalling for other reasons, so only 
when this discussion resurfaced, I started thinking about this again, 
and discovered that I didn't have an answer.

That is, in the description

a=group:bundle foo bar baz
m=video 4270
a=mid:foo
a=property-x:1
m=video
a=mid:bar
a=property-x:2
m=video
a=mid:baz
a=property-x:3

one would expect a single RTP stream to be set up; when the recipient 
gets 3 video streams with SSRCS 234, 235 and 236, how does he know which 
stream should have which value for "property-x"?
One can solve it with SSRC-associated signalling in this case too:

a=ssrc:234 property-x:1

but then you're back to doing SSRC-associated signalling, and if you're 
already doing that, we're not very different from the MMT description 
anyway.

That's how I think of it at the moment, anyway - YMMV.

---

From: "Stach, Thomas" <[email-masked]>
Date: Thu, 11 Oct 2012 14:13:23 +0000

Harald,

thanks for the reply.
It helps for my second question, but not for my main question.
How can the answerer recognize (from the m=anymedia line alone) that multiple video streams need to be set up in the same RTP session?

Regards
Thomas  

> -----Ursprüngliche Nachricht-----
> Von: Harald Alvestrand [mailto:[email-masked]] 
> Gesendet: Donnerstag, 11. Oktober 2012 15:41
> An: Stach, Thomas
> Cc: Christer Holmberg; Jonathan Lennox; mmusic WG
> Betreff: Re: How does mmt grouping work with multiple streams 
> of the same type?
> 
> > I have the impression that the 
> draft-holmberg-mmusic-sdp-mmt-negotiation is based on 
> Jonathan's proposal
> > * The m=bundle is a complete description of an entirely 
> separate m-line; if it's used, the other m-lines in the group 
> are rejected and the information they carry is ignored.
> >  From the currently proposed syntax for the m=anymedia line 
> I cannot deduce that multiple audio or video streams are offered.
> > In case that e.g. the video stream use different codecs, I 
> could also learn that only from the legacy m-lines.
>

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 11 Oct 2012 16:44:47 +0200

> How can the answerer recognize (from the m=anymedia line alone) that multiple video streams need to be set up in the same RTP session?
If each video stream is signalled by the presence of SSRC-associated 
signalling for that video stream, you count the number of video streams 
signalled.

---

From: "Stach, Thomas" <[email-masked]>
Date: Thu, 11 Oct 2012 14:55:11 +0000

inline

> Gesendet: Donnerstag, 11. Oktober 2012 16:45
> An: Stach, Thomas
> Betreff: Re: AW: How does mmt grouping work with multiple 
> streams of the same type?
> 
> > How can the answerer recognize (from the m=anymedia line 
> alone) that multiple video streams need to be set up in the 
> same RTP session?
> signalling for that video stream, you count the number of 
> video streams signalled.

Could that be done by e.g. enhancing the new a=mmtype with an ssrc element?
E.g.

       a=mmtype: 0 audio ssrc=1
       a=mmtype: 8 audio ssrc=1
       a=mmtype: 97 audio ssrc=2
       a=mmtype: 31 video ssrc=3
       a=mmtype: 32 video ssrc=4 

would mean 2 audio streams and 2 video streams, whereas 

       a=mmtype: 0 audio ssrc=1
       a=mmtype: 8 audio ssrc=1
       a=mmtype: 97 audio ssrc=1
       a=mmtype: 31 video ssrc=2
       a=mmtype: 32 video ssrc=2 

means 1 audio and 1 video stream based on the number of different SSRCs?

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 11 Oct 2012 17:15:55 +0200

> means 1 audio and 1 video stream based on the number of different SSRCs?

On the principle of "don't invent new stuff if you already have stuff", 
I think we should continue to use RFC 5576 for this purpose:

a=ssrc:1 <something follows>

The suggestion above wouldn't work for the case of two SSRCs both using 
the same payload type.

---

From: "Stach, Thomas" <[email-masked]>
Date: Fri, 12 Oct 2012 07:58:01 +0000

inline

> Gesendet: Donnerstag, 11. Oktober 2012 17:16
> An: Stach, Thomas
> Betreff: Re: AW: AW: How does mmt grouping work with multiple 

I haven't thought about that, but yes it would work for as well.
For two audio streams (audio1 w/ PCMU or PCMA, audio2 w/ iLBC) it would look like this? 

   a=rtpmap:0 PCMU/8000
   a=rtpmap:8 PCMA/8000
   a=rtpmap:97 iLBC/8000
   a=mmtype: 0 audio 
   a=mmtype: 8 audio 
   a=mmtype: 97 audio
   a=ssrc:12345 fmtp:0 
   a=ssrc:12345 fmtp:8 
   a=ssrc:67890 fmtp:97 

> 
> The suggestion above wouldn't work for the case of two SSRCs 
> both using the same payload type.
Well it would if you duplicated the a=mmtype attribute with different ssrc elements in that case.
     a=mmtype: 0 audio ssrc=1
     a=mmtype: 0 audio ssrc=2
Would you do something else for a=ssrc?

Regards
Thomas

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 12 Oct 2012 10:44:35 +0200

> For two audio streams (audio1 w/ PCMU or PCMA, audio2 w/ iLBC) it would look like this?
Yes, except that as far as I can tell from RFC 5576, you can't use fmtp: 
without an actual parameter value: "This parameter MUST only be used for 
media
    types for which source-level format parameters have explicitly been
    specified". (section 6.3).

If you need to use a field that's always present, use CNAME.

Note(1): If you only use one payload type for all your audio, you have 
no need to signal which format they're using.

Note(2): In the above, you have ssrc 12345 using both PCMU/8000 and 
PCMA/8000. I don't know if it's intentional or not.

> Would you do something else for a=ssrc?

Well, a=ssrc *is* something else. The above mixes info about payload 
types and info about SSRCs; I think that's a bad design.

---

From: "Stach, Thomas" <[email-masked]>
Date: Fri, 12 Oct 2012 10:15:19 +0000

> PCMA/8000. I don't know if it's intentional or not.
Yes, this was intentional, 
I used two different ssrc (12345, 67890) in order to indicate that I want to establish two audio streams. 
I used ssrc=12345 twice in order to give a hint that the first audio stream should either use PCMU or PCMA (and ssrc=12345).
The second audio stream would use ssrc=67890 and iLBC.

However from your responses I'm not sure if we are on the same page with respect to what I want to achieve.
Let me tray to clarify.

My understanding is that 
- the m=anymedia gives a complete description of what is offered,
- if the answerer understands m=anymedia, it accepts the m=anymedia line and rejects all other m-lines
- if the answerer understands m=anymedia, it can ignore all other m-lines in the SDP, it doesn't even have to look at them in order to build its answer.

If this is correct, consider an SDP offer with e.g. 4 m-lines (2 audio+2 video)

       v=0
       o=alice 2890844526 2890844526 IN IP4 host.atlanta.com
       s=
       c=IN IP4 host.atlanta.com
       t=0 0
       m=audio 10000 RTP/AVP 0 8
       a=rtpmap:0 PCMU/8000
       a=rtpmap:8 PCMA/8000
       m=audio 10002 RTP/AVP 97
       a=rtpmap:97 iLBC/8000
       m=video 20000 RTP/AVP 31 
       a=rtpmap:31 H261/90000
       m=video 20002 RTP/AVP 32
       a=rtpmap:32 MPV/90000

How would the associated m=anymedia line look like in this case, such that the answerer can see 
that 2 audio and 2 video streams are offered without having to look into the other m-lines?

How would the answer look like?

Thanks
Thomas

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 12 Oct 2012 12:41:28 +0200

Hi,

Jumping in late - trying to catch up :)

>> without an actual parameter value: "This parameter MUST only be used 
>> for media types for which source-level format parameters have explicitly 
>> been specified". (section 6.3).
>> 

Even without BUNDLE, this is how you would do it if you offer multiple streams within a single m- line.

> - if the answerer understands m=anymedia, it accepts the m=anymedia line and rejects all other m-lines

Correct.

(All other m- lines associated with the MMT group, that is)

> - if the answerer understands m=anymedia, it can ignore all other m-lines in the SDP, it doesn't even have to look at them in order to build its answer.

Correct.

(All other m- lines associated with the MMT group, that is)

> How would the associated m=anymedia line look like in this case, such that the answerer can see that 2 audio and 2 video streams are offered without having to look into the other m-lines?

Note that there are still some details (e.g. whether "RTP/AVP" is going to be used) that have to be sorted out, but something like this:

m=anytype 10000 RTP/AVP 0 8 97 31 32
a=mmtype: 0 audio
a=mmtype: 8 audio
a=mmtype: 97 audio
a=mmtype: 31 video
a=mmtype: 32 video
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 iLBC/8000
a=rtpmap:31 H261/90000
a=rtpmap:32 MPV/90000

Then, you will use the SSRC attribute to identify the streams. But, we do need to define how to associate SSRCs with media types and/or payload types.

In one of your e-mails, you suggested to extend the mmtype attribute. That's for sure one possibility.

Regards,

Christer

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 12 Oct 2012 12:41:54 +0200

> that 2 audio and 2 video streams are offered without having to look into the other m-lines?
This offer does not guarantee that there are only 2 audio and 2 video 
streams.
It makes it a fair assumption that there are at least 2 of each (unless 
some of the m-lines are intended to be used for receiveonly), but it is 
perfectly legal to send more than one audio stream in each RTP session 
under each M-line.

So I don't think the starting point you're starting from is correct; 
until draft-westerlund-mmusic-max-ssrc or something like it is accepted, 
you have no way to tell *now* how many streams you are going to get.

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 12 Oct 2012 12:43:46 +0200

On 10/12/2012 12:41 PM, Christer Holmberg wrote:
> Then, you will use the SSRC attribute to identify the streams. But, we 
> do need to define how to associate SSRCs with media types and/or 
> payload types. In one of your e-mails, you suggested to extend the 
> mmtype attribute. That's for sure one possibility. Regards, Christer 
If you need to know this before you start getting media, we need signalling.

Otherwise, we just look at the PT field and the SSRC field in the 
incoming packets.

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 12 Oct 2012 13:06:44 +0200

Hi,

>> mmtype attribute. That's for sure one possibility. 
>
> Otherwise, we just look at the PT field and the SSRC field in the incoming packets.

Correct.

I guess we can add it as an open issue whether one needs to be able to signal it.

But, again, the possibility to associate PTs with SSRCs is not BUNDLE specific. You may want it also when you have multiple streams (same media type) associated with a single m- line.

Regards,

Christer

---

From: "Stach, Thomas" <[email-masked]>
Date: Fri, 12 Oct 2012 12:01:44 +0000

Christer Holmberg wrote:
>>> Yes, except that as far as I can tell from RFC 5576, you can't use
>>> fmtp: without an actual parameter value: "This parameter MUST only
>>> be used for media types for which source-level format parameters
>>> have explicitly been specified". (section 6.3).
>>> 
>>> Note(1): If you only use one payload type for all your audio, you
>>> have no need to signal which format they're using.
>>> 
>> I used two different ssrc (12345, 67890) in order to
> indicate that I want to establish two audio streams.
>> 
>> I used ssrc=12345 twice in order to give a hint that the
> first audio stream should either use PCMU or PCMA (and ssrc=12345).
> Even without BUNDLE, this is how you would do it if you offer
> multiple streams within a single m- line.
> 
>> However from your responses I'm not sure if we are on the
> same page with respect to what I want to achieve.
>> Let me tray to clarify.
>> 
>> My understanding is that
>> - if the answerer understands m=anymedia, it accepts the
> m=anymedia line and rejects all other m-lines
> 
> Correct.
> 

Of course

> 
>> - if the answerer understands m=anymedia, it can ignore all
> other m-lines in the SDP, it doesn't even have to look at
> them in order to build its answer.
> 
> Correct.
> 
Yes, this is more precise again

>>       o=alice 2890844526 2890844526 IN IP4 host.atlanta.com       s=
>> How would the associated m=anymedia line look like in this
> case, such that the answerer can see that 2 audio and 2 video
> streams are offered without having to look into the other m-lines?
> 
> Note that there are still some details (e.g. whether
> "RTP/AVP" is going to be used) that have to be sorted out,
> but something like this:
> 
Yes, something like this. But this m=anymedia line could also be based on:

SDP Offer (1 audio + 1 video)
       v=0
       o=alice 2890844526 2890844526 IN IP4 host.atlanta.com
       s=
       c=IN IP4 host.atlanta.com
       t=0 0
       m=audio 10000 RTP/AVP 0 8 97
       a=rtpmap:0 PCMU/8000
       a=rtpmap:8 PCMA/8000
       a=rtpmap:97 iLBC/8000
       m=video 20000 RTP/AVP 31 32
       a=rtpmap:31 H261/90000
       a=rtpmap:32 MPV/90000

I think the a=anymedia line in the offer needs provide a means to dinstinguish this from the case above.

> Then, you will use the SSRC attribute to identify the
> streams. But, we do need to define how to associate SSRCs
> with media types and/or payload types.
> 

Thanks, 

once available I'll look into the updated draft on how you intend to solve that.

Regards
Thomas

> In one of your e-mails, you suggested to extend the mmtype
> attribute. That's for sure one possibility.
> 
> Regards,
> 
> Christer

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 12 Oct 2012 14:16:58 +0200

Hi,

>> case, such that the answerer can see that 2 audio and 2 video streams 
>> are offered without having to look into the other m-lines?
>> 
>> Note that there are still some details (e.g. whether "RTP/AVP" is 
>> going to be used) that have to be sorted out, but something like this:
>> 

When I look at your two offer examples, I see the following differences:

1. The port numbers. That is obviously nothing you need to distinguish, as the same port will be used for everything when using m=anytype.

2. RTP sessions. Assuming that each m- line represents an RTP session, and you want to keep that within the m=anytype description, that is obviously something that needs to be defined. However, the basic assumption in BUNDLE is that all multiplexed media belongs to the same RTP sessions, and you'll need some other mechanism/extension (e.g. the SHIM mechanism) in order to use multiple RTP sessions.

Assuming you will use the SSRC attribute to indicate the number of streams, what else do you need to distinguish?

>> payload types.

You are very welcome to present ideas on how to solve it :)

Regards,

Christer

---

From: "Stach, Thomas" <[email-masked]>
Date: Fri, 12 Oct 2012 12:35:12 +0000

> 
> Assuming you will use the SSRC attribute to indicate the
> number of streams, what else do you need to distinguish?
With the additional a=ssrc attributes it will be sufficient.

Thanks
Thomas