Subject: Re: [MMUSIC] M-lines: Wikifying it

From: Harald Alvestrand <[email-masked]>
Date: Sat, 01 Dec 2012 16:00:32 +0100

On 11/28/2012 09:13 PM, Paul Kyzivat wrote:
> Inline
>
> On 11/28/12 9:50 AM, Harald Alvestrand wrote:
>> Perhaps we can do some real work rather than just disagreeing on 
>> principles.
>>
>> Perhaps the most contentious issue now is "what should an M-line be".
>> As far as I can tell, M-lines and their parameters describe 3 things:
>>
>> - Properties of the transport flow / RTP session (Transport, or T for 
>> short)
>> - Properties that make sense to apply to an individual media stream, and
>> may need to be different for different media streams in the same
>> combined RTP session (Flow, or F for short)
>> - Properties that make sense to apply to all streams of a given type
>> (Media, or M for short)
>>
>> The properties that can be represented in M-lines are found in two IANA
>> registries, both on
>> http://www.iana.org/assignments/sdp-parameters/sdp-parameters.xml:
>>
>> - att-field (both session and media level)
>> - att-field (media level only)
>>
>> I make it 179 entries in these categories, combined.
>>
>> I see the same kind of work being necessary no matter which way we go:
>>
>> - If we go with M-lines describing a whole RTP session (the MMT
>> proposal), each property of the "F" class that we stil need needs to be
>> copied onto a property in the "att-field (source level)" registry.
>>
>> - If we go with M-lines describing groups of media streams, degenerating
>> to a single media stream in the case of using "a=inactive" for stream
>> muting (the BUNDLE and TOGETHER proposals), each property of the "T"
>> class needs to be described as "MUST have consistent values for all the
>> combined M-lines", "MUST be taken only from the first M-line in the
>> bundle argument", or "must get special treatment".
>>
>> - In both cases, the "M"-class lines don't need any special treatment.
>> They just need to be present in all the M-lines where they're relevant.
>>
>> I've created a spreadsheet here:
>>
>> https://docs.google.com/spreadsheet/ccc?key=0Av-aF69sk5C6dDhPblpRV2pkbXVGNV9lUFI3cHlIRFE 
>>
>>
>> with 2 extra columns: "Class" (M, F or T) and "Importance" (1 for
>> "necessary" and 0 for "will never matter").
>>
>> Anyone should be able to write it.
>> If a few of us do a quick job of markup on it, I think we should be able
>> to have a more realistic assessment in a few days.
>>
>> I've started, but much of this is unknown to me. Can others help?
>
> Don't forget the media formats listed in the m-line itself. The 
> meaning of these depends on the <proto> param. The focus of this 
> discussion is on those <proto> values that correspond to various 
> flavors of RTP. There are a bunch of <proto> values that don't 
> correspond to RTP, and that assign different meanings to the format 
> parameters.
I'm happy to ignore anything but RTP. BUNDLE isn't defined for anything 
else (while, in the parallel discussion, we may get DTLS/SCTP within its 
scope too, the RTP parameter set is our current problem).

>
> AFAIK, of the currently defined proto values only the RTP-related ones 
> have the notion of multiple flows within the session. But the sdp 
> extensions for SCTP media are currently being defined. That certainly 
> has the potential to be considered as multiple flows, and should be 
> considered in this discussion.
>
> For RTP transports, the formats are payload type numbers, and these 
> are mapped to actual formats (codecs) via a=rtpmap. I see that you 
> have proposed that rtpmap be per-media, not transport or flow. I guess 
> this means that the same payload type number could mean one thing for 
> all audio flows over the transport, and something different for all 
> video flows over the same transport.
Absolutely not; that would violate both common sense and the BUNDLE 
draft as specified. The BUNDLE drafts explicitly say that a payload type 
number can only be used for one thing.
> But the payload numbers to be used (specified on the m-line) is 
> negotiated in O/A. How would that work for audio vs. video? ISTM it 
> would be preferred to negotiate payload numbers and types 
> independently per flow.
See above. They can't collide.
>
> Regarding the "importance" column in the table: I don't understand how 
> you have assigned values. In what frame of reference is importance to 
> be assessed?
My completely subjective opinion, for RTCWEB. If others disagree, please 
change and add a comment. If we all agree, it becomes consensus.
>
> Regarding a=label: This could be any of T/F/M, depending on 
> application use. I think we must assume T, and consider an alternative 
> for F. (And for M if we think it is needed.)
The claim that a=label can be used to mark flows makes it F. If the 
people making that claim are willing to let go of the claim, we can 
change it to T. But at the moment its proponents are clearly calling for F.
>
> (I'm not convinced we need to identify an "M" scope distinct from T 
> and F.)
Please consider the ones marked as such and tell me the argument for why 
it's obviously one or the other.

In particular, a=rtpmap:<number> is clearly not T, but when we have 
multiple flows on an RTP session utilizing the same codec, I'd hate to 
be forced to create multiple identical payload types just because it was 
classified as an F field.

>
>     Thanks,
>     Paul
> _______________________________________________
> mmusic mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmusic

---

From: Paul Kyzivat <[email-masked]>
Date: Sat, 01 Dec 2012 14:54:13 -0500

inline

> scope too, the RTP parameter set is our current problem).

Hmm. OK, I went looking for something that says that. Section 7.1 does 
say that, though I can find no text that talks about which m-line 
<proto> values may be bundled together.

> change and add a comment. If we all agree, it becomes consensus.

Frankly I don't even know what most of the parameters in the list are.
But I assume that most of them were at one time important to somebody - 
at least those who defined it. Perhaps some are now obsolete, but most 
likely not all of them.

It isn't helpful if we all agree on what is important and what is not if 
we don't agree on the criterion for importance.

A couple of meanings that could be useful here:

- those that might be used in an rtcweb context vs. those that nobody
   thinks make any sense to use with rtcweb

- those that make sense to use in a bundle/mmt vs. those that may only
   be used outside a bundle/mmt.

I guess I would prefer the latter. It has the benefit that those nobody 
understands can be put in the unimportant category, and they won't be 
broken by bundle/mmt.

>> for F. (And for M if we think it is needed.)

I may have misunderstood your intended meaning for M. I assume you meant 
all RTP streams of the same media type (audio or video) within a single 
5-tuple.

> The claim that a=label can be used to mark flows makes it F.

a=label is a media-level attribute, so it is bound to an m-line.

In the absence of bundle, regardless of whether the corresponding RTP 
session has one or several flows. The m-line has a particular media 
type, so I guess it could be viewed as either T or M.

Some applications of a=label may assume or require that the labeled 
m-line contain only one RTP stream. If so, then perhaps the label could 
be considered to be F. But we can't really know that.

> If the

Is your point that you know of cases where there is a desire to label an 
individual RTP-stream, and you would like to have a way to do that when 
RTP-streams are multiplexed in an RTP session?

I'm ok with that.

> classified as an F field.

a=rtpmap is the *only* one currently marked M. And ISTM that it clearly 
is a T.

I say this because, as you have said elsewhere, each payload type number 
is tied to a particular codec of a particular codec, whether audio or 
video. It isn't possible to have payload type number 100 mean one thing 
for audio and a different thing for video.

OTOH, when multiplexing audio and video on the same 5-tuple it is 
necessary to know whether a particular a=rtpmap is intended to be audio 
or video. Normally it inherits that from the m-line it belongs to. That 
still works for bundle. It doesn't work for mmt.

So, I still think there is no need for M. But a=rtpmap is "special" in 
that it requires an extra datum. Unless there are other attributes that 
have the same issue I think this is just a special case.

	Thanks,
	Paul

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 3 Dec 2012 10:23:28 +0000

Hi,

A question for clarification:

When talking about the "F" class, what do you mean by "stream"? Do you mean individual sources associated with the m- line? Or, do you mean all sources associated with the m- line?

I ASSUME you mean all sources associated with the m- line, because otherwise you will need the a=ssrc attribute.

Regards,

Christer

From: [email-masked] [mailto:[email-masked]] On Behalf Of Harald Alvestrand
Sent: 28. marraskuuta 2012 16:51
To: mmusic
Subject: [MMUSIC] M-lines: Wikifying it

Perhaps we can do some real work rather than just disagreeing on principles.

- Properties of the transport flow / RTP session (Transport, or T for short)
- Properties that make sense to apply to an individual media stream, and may need to be different for different media streams in the same combined RTP session (Flow, or F for short)
- Properties that make sense to apply to all streams of a given type (Media, or M for short)

The properties that can be represented in M-lines are found in two IANA registries, both on
http://www.iana.org/assignments/sdp-parameters/sdp-parameters.xml:

- If we go with M-lines describing a whole RTP session (the MMT proposal), each property of the "F" class that we stil need needs to be copied onto a property in the "att-field (source level)" registry.

- If we go with M-lines describing groups of media streams, degenerating to a single media stream in the case of using "a=inactive" for stream muting (the BUNDLE and TOGETHER proposals), each property of the "T" class needs to be described as "MUST have consistent values for all the combined M-lines", "MUST be taken only from the first M-line in the bundle argument", or "must get special treatment".

- In both cases, the "M"-class lines don't need any special treatment. They just need to be present in all the M-lines where they're relevant.

with 2 extra columns: "Class" (M, F or T) and "Importance" (1 for "necessary" and 0 for "will never matter").

Anyone should be able to write it.
If a few of us do a quick job of markup on it, I think we should be able to have a more realistic assessment in a few days.

                      Harald

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 03 Dec 2012 11:53:32 +0100

On 12/03/2012 11:23 AM, Christer Holmberg wrote:
> When talking about the "F" class, what do you mean by "stream"? Do you 
> mean individual sources associated with the m- line? Or, do you mean 
> all sources associated with the m- line?
>
> I ASSUME you mean all sources associated with the m- line, because 
> otherwise you will need the a=ssrc attribute.
>

The purpose of the exercise is to figure out what attributes we need to 
redefine if we define the M-line to be an RTP session (the MMT and the 
all-videos-under-one-M-line approach) as opposed to what attributes we 
need to redefine if we define the M-line to be a group of sources that, 
in turn, may be associated with an RTP session (the BUNDLE approach).

F-lines need redefinition under the first approach, T-lines need 
redefinition (or adjustment) under the second, M-lines may be able to 
work in both cases.

> Regards,
>
> Christer
>
> *From:[email-masked] [mailto:[email-masked]] *On 
> Behalf Of *Harald Alvestrand
> *Sent:* 28. marraskuuta 2012 16:51
> *To:* mmusic
> *Subject:* [MMUSIC] M-lines: Wikifying it
>
> - Properties that make sense to apply to an individual media stream, 
> and may need to be different for different media streams in the same 
> proposal), each property of the "F" class that we stil need needs to 
> be copied onto a property in the "att-field (source level)" registry.
>
> - If we go with M-lines describing groups of media streams, 
> degenerating to a single media stream in the case of using 
> "a=inactive" for stream muting (the BUNDLE and TOGETHER proposals), 
> each property of the "T" class needs to be described as "MUST have 
> consistent values for all the combined M-lines", "MUST be taken only 
> from the first M-line in the bundle argument", or "must get special 
> treatment".
>
> If a few of us do a quick job of markup on it, I think we should be 
> able to have a more realistic assessment in a few days.
>

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 3 Dec 2012 11:05:49 +0000

Hi,

So, the exercise is NOT about how to apply attributes to individual sources?

Regards,

Christer

From: Harald Alvestrand [mailto:[email-masked]]
Sent: 3. joulukuuta 2012 12:54
To: Christer Holmberg
Cc: mmusic
Subject: Re: [MMUSIC] M-lines: Wikifying it

The purpose of the exercise is to figure out what attributes we need to redefine if we define the M-line to be an RTP session (the MMT and the all-videos-under-one-M-line approach) as opposed to what attributes we need to redefine if we define the M-line to be a group of sources that, in turn, may be associated with an RTP session (the BUNDLE approach).

F-lines need redefinition under the first approach, T-lines need redefinition (or adjustment) under the second, M-lines may be able to work in both cases.

Regards,

Christer

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Harald Alvestrand
Sent: 28. marraskuuta 2012 16:51
To: mmusic
Subject: [MMUSIC] M-lines: Wikifying it

                      Harald

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 03 Dec 2012 12:06:46 +0100

On 12/01/2012 08:54 PM, Paul Kyzivat wrote:
>>>> degenerating
>>>> able
Section 7.2 of draft-ietf-mmusic-sdp-bundle says that:

    o  - The "proto" value in each "m=" line MUST be identical (e.g.
       RTP/AVPF).

> But I assume that most of them were at one time important to somebody 
> - at least those who defined it. Perhaps some are now obsolete, but 
> most likely not all of them.
>
> It isn't helpful if we all agree on what is important and what is not 
> if we don't agree on the criterion for importance.
>
> I guess I would prefer the latter. It has the benefit that those 
> nobody understands can be put in the unimportant category, and they 
> won't be broken by bundle/mmt.
I changed the column title to "Importance to RTCWEB". Feel free to add 
new columns if you feel they are useful.
> I may have misunderstood your intended meaning for M. I assume you 
> meant all RTP streams of the same media type (audio or video) within a 
> single 5-tuple.
I was thinking "same payload type", since the payload type is the 
distinguisher used for groups of flows that should have the same 
parameters in many cases.
> type, so I guess it could be viewed as either T or M.
The reason why we're having the discussion is that we have two competing 
definitons of what an M-line should be, and need to choose one.
So arguing from whether or not something associates with an M-line is 
not useful.
> m-line contain only one RTP stream. If so, then perhaps the label 
> could be considered to be F. But we can't really know that.
What applications of a=label do you know about?
> Is your point that you know of cases where there is a desire to label 
> an individual RTP-stream, and you would like to have a way to do that 
> when RTP-streams are multiplexed in an RTP session?

Yes, that's how I've interpreted statements made (mostly from Cullen) 
about a=label.

> a=rtpmap is the *only* one currently marked M. And ISTM that it 
> clearly is a T.
The a=imageattr attribute is also indexed by payload type. I just 
haven't gotten to it yet.

>
> I say this because, as you have said elsewhere, each payload type 
> number is tied to a particular codec of a particular codec, whether 
> audio or video. It isn't possible to have payload type number 100 mean 
> one thing for audio and a different thing for video.
>
> necessary to know whether a particular a=rtpmap is intended to be 
> audio or video. Normally it inherits that from the m-line it belongs 
> to. That still works for bundle. It doesn't work for mmt.
>
> that it requires an extra datum. Unless there are other attributes 
> that have the same issue I think this is just a special case.
>
>     Thanks,
>     Paul
>

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 03 Dec 2012 12:14:40 +0100

> So, the exercise is NOT about how to apply attributes to individual 

Only in the sense that it is about figuring out which attributes need to 
be applied to individual sources.

> Regards,
>
> Christer
>
> *From:*Harald Alvestrand [mailto:[email-masked]]
> *Sent:* 3. joulukuuta 2012 12:54
> *To:* Christer Holmberg
> *Cc:* mmusic
> *Subject:* Re: [MMUSIC] M-lines: Wikifying it
>
>     When talking about the "F" class, what do you mean by "stream"? Do
>     you mean individual sources associated with the m- line? Or, do
>     you mean all sources associated with the m- line?
>
> The purpose of the exercise is to figure out what attributes we need 
> to redefine if we define the M-line to be an RTP session (the MMT and 
> the all-videos-under-one-M-line approach) as opposed to what 
> attributes we need to redefine if we define the M-line to be a group 
> of sources that, in turn, may be associated with an RTP session (the 
> BUNDLE approach).
>
> *From:[email-masked] <mailto:[email-masked]> 
> [mailto:[email-masked]] *On Behalf Of *Harald Alvestrand

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 3 Dec 2012 11:16:41 +0000

Hi,
 
>Only in the sense that it is about figuring out which attributes need to be applied to individual sources.

Well, I believe that was my original question :)

If you want to apply an attribute to an individual source, you must use the ssrc attribute. 
 
Regards,
 
Christer 

 

 
Regards,
 
Christer

                      Harald

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 03 Dec 2012 16:48:53 +0100

> If you want to apply an attribute to an individual source, you must use the ssrc attribute.

If you are in the camp that believes that there should be more than one 
independent source per M-line.
That camp seemed to me to be in the minority in Atlanta; it was 
absolutely not the consensus opinion.

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 3 Dec 2012 16:12:33 +0000

Hi,

> If you are in the camp that believes that there should be more than oneindependent source per M-line.
>
> That camp seemed to me to be in the minority in Atlanta; it was absolutely not the consensus opinion.

It's not about in which camp someone is, it's about the assumptions we make in the wiki study. If the assumption is one source per m- line, than I think it should be explicitly stated.

Regards,

Christer

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 03 Dec 2012 17:38:22 +0100

> It's not about in which camp someone is, it's about the assumptions we make in the wiki study. If the assumption is one source per m- line, than I think it should be explicitly stated.
Trying again, since I'm obviously not making myself clear:

The two alternative approaches are:

- Everything that needs to be controlled separately is in its own 
M-line. Everything that needs to go on the same RTP session uses BUNDLE 
to get there. Transport parameters get figured out somehow.

- Everything that goes on the same RTP session goes on the same M-line. 
Due to backwards compatible SDP, we may choose to use BUNDLE to lump 
video and audio together - or we may choose MMT if that works for 
people. Anything that needs separate control uses a=ssrc.

In Atlanta, the room was clearly split between the two. I'm trying to 
get real data on how many things need to be redefined for each case.

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 3 Dec 2012 16:44:51 +0000

Hi, 

> - Everything that needs to be controlled separately is in its own M-line. Everything that needs to go on the same RTP session uses BUNDLE to get there. Transport parameters get figured out somehow.
>
> Due to backwards compatible SDP, we may choose to use BUNDLE to lump video and audio together - or we may choose MMT if that works for people. Anything that needs separate control uses a=ssrc.

Exactly!

So, which (if any) approach do you assume in the wiki? :)

Regards,

Christer

> In Atlanta, the room was clearly split between the two. I'm trying to get real data on how many things need to be redefined for each case.

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 03 Dec 2012 17:59:11 +0100

> So, which (if any) approach do you assume in the wiki? :)
For the 5th time: I'm NOT assuming one or the other approach.
I'm trying to gather information that people can use to decide which 
approach to follow.
That REQUIRES that the wiki does NOT assume one approach.

Do you hear me now?

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 3 Dec 2012 17:18:46 +0000

Hi, 

> I'm trying to gather information that people can use to decide which approach to follow.

So, coming back to my original question: For "F", a *stream* refers to an individual source (and is not related to whether one m- line will or will not contain multiple sources). Correct?

Regards,

Christer

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 03 Dec 2012 16:37:52 -0500

> What applications of a=label do you know about?

I know of two: BFCP and SIPREC.

In BFCP the label is used to associate a bfcp floor with an m-line. I 
don't think it matters whether that m-line carries one or several RTP 
streams.

We can speculate how we would want this to work in the case that several 
media m-lines were bundled together. I don't know - too many hypotheticals.

In SIPREC, the label is used to associate an m-line with certain 
metadata that describes the source of the media being sent via that 
m-line. The siprec group has intentionally kept the question of whether 
the content in that m-line is a single RTP stream or if it can be 
several multiplexed together. There is nothing to forbid multiplexing, 
but no specification of what it means either.

SIPREC *might* have specified use of bundle, or explicitly specified 
multiplexing of RTP streams over a single m-line, if those things were 
better defined. But how it might eventually use bundle or mmt is again a 
hypothetical right now.

A problem for siprec is that it is recording media from existing sip 
sessions, without any knowledge of how they use SDP and RTP. We could 
find it recording a call initiated with RTCWEB that uses bundle or mmt. 
Most likely siprec will need to be extended to support that, though I 
wish that wasn't so.

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 03 Dec 2012 16:44:32 -0500

> absolutely not the consensus opinion.

There were certainly competing camps supporting bundle and mmt.
It was not my understanding that supporting bundle means believing that 
each source gets its own m-line.

(I was going to check the mmusic minutes, but they aren't posted yet.)

	Thanks,
	Paul

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 4 Dec 2012 08:08:42 +0000

Hi,

>>>> Only in the sense that it is about figuring out which attributes 
>>>> need to be applied to individual sources.
>> If you are in the camp that believes that there should be more than 
>> one independent source per M-line.
> It was not my understanding that supporting bundle means believing that each source gets its own m-line.

Correct.

However, I think the "BUNDLE vs MMT" issue is separate from the "single- vs multiple source per m- line" issue, which I guess is what Harald was referring to.

Regards,

Christer

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 04 Dec 2012 10:22:45 +0100

> So, coming back to my original question: For "F", a *stream* refers to an individual source (and is not related to whether one m- line will or will not contain multiple sources). Correct?
I don't want to mix in the "source" word (CLUE has talked about sources 
that originate multiple streams enough to confuse that term), but if you 
want to think in terms of sources, yes.