Subject: [siprec] What does an MS object represent? Can we pick one of these options?

From: "Elwell, John" <[email-masked]>
Date: Mon, 23 May 2011 11:18:48 +0200

(Starting a new thread, because the previous thread ("Does MS have lifetime...") was getting confused in its various branches.)

I think the fundamental question at issue is what exactly an MS object represents. Draft-00 is certainly unclear on this. I think we have been discussing two possibilities:

1. An MS object represents a given medium (e.g., audio, main video, presentation video, current speaker video...). Different participants can contribute to this medium, in which case they are all contributing to this single MS object. For example, in a 2-party audio-only CS, there is a single MS object (audio) contributed to by two participants. This has been my way of viewing it, but certainly not everyone considers it this way, and I am prepared to be persuaded otherwise.

2. An MS object represents a given medium as contributed to by a single participant (or by multiple participants that are not separately identifiable and/or where their contributions are inseparable). Where different participants are contributing to the same medium, this is represented by multiple MS objects. For example, in a 2-party audio-only call, there are two MS objects. In this case, presumably there would need to be some mechanism for tying together those MS objects representing the same medium (e.g., the two MS objects representing audio in this same example). Possibly the name Media Stream is not sufficiently descriptive in this case, but I can't come up with a better name.

I think we can possibly make this basic decision, and then fill in the details afterwards.

John (as individual)

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 23 May 2011 08:22:39 -0400

John,

I have been thinking of it as more like (2). I would allow the two 
streams matched in an o/a by m-line offset to be reported as two MS if 
passed to the RS separately, as as one MS if mixed and passed to the RS 
as one.

I guess you are right that we ought to indicate the pairing of MS in the 
sense above via something in the metadata. If not explicitly so 
indicated it can probably be inferred in common simple cases, but might 
be difficult to infer in more complex cases.

It is good when we identify and sort out these ambiguities.

	Thanks,
	Paul

On 5/23/2011 5:18 AM, Elwell, John wrote:
> _______________________________________________
> siprec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/siprec
>

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Mon, 23 May 2011 08:53:52 -0700

My thinking up until this point has been more aligned with option 2.
I so see some benefit and simplification provided by option 1, but I
think the flexibility provided by option 2 is worth the added complexity
of representing an MS on a more granular level.

Cheers,
Charles

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
Behalf Of Elwell, John
> Sent: Monday, May 23, 2011 2:19 AM
> To: [email-masked]
> Subject: [siprec] What does an MS object represent? Can we pick one of
theseoptions?
> 
> (Starting a new thread, because the previous thread ("Does MS have
lifetime...") was getting confused
> in its various branches.)
> 
> I think the fundamental question at issue is what exactly an MS object
represents. Draft-00 is
> certainly unclear on this. I think we have been discussing two
possibilities:
> 
> 1. An MS object represents a given medium (e.g., audio, main video,
presentation video, current
> speaker video...). Different participants can contribute to this
medium, in which case they are all
> contributing to this single MS object. For example, in a 2-party
audio-only CS, there is a single MS
> object (audio) contributed to by two participants. This has been my
way of viewing it, but certainly
> not everyone considers it this way, and I am prepared to be persuaded
otherwise.
> 
> 2. An MS object represents a given medium as contributed to by a
single participant (or by multiple
> participants that are not separately identifiable and/or where their
contributions are inseparable).
> Where different participants are contributing to the same medium, this
is represented by multiple MS
> objects. For example, in a 2-party audio-only call, there are two MS
objects. In this case, presumably
> there would need to be some mechanism for tying together those MS
objects representing the same medium
> (e.g., the two MS objects representing audio in this same example).
Possibly the name Media Stream is
> not sufficiently descriptive in this case, but I can't come up with a
better name.

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 23 May 2011 12:23:14 -0400

IIUC, option 1 can be achieved as one of the possibilities within option 2.

	Thanks,
	Paul

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Mon, 23 May 2011 11:02:07 -0700

Essentially yes, though option 1 and option 2 are just two different
representations of the information. I think it is really a matter of
which way we think is more straightforward. If you want to be able to
break out the media on a per participant basis, it is more intuitive, in
my opinion, to go with option 2.

Cheers,
Charles

> From: [email-masked] [mailto:[email-masked]] On
Behalf Of Paul Kyzivat (pkyzivat)
> Sent: Monday, May 23, 2011 9:23 AM
> To: [email-masked]
> Subject: Re: [siprec] What does an MS object represent? Can we pick
one of theseoptions?
> >> I think we can possibly make this basic decision, and then fill in
the
> > details afterwards.

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 23 May 2011 16:58:11 -0400

My goal here is to leave the SRC with as much flexibility as possible 
about how it does things. 2 provides more options than 1 does.

	Thanks,
	Paul

---

From: "Parthasarathi R (partr)" <[email-masked]>
Date: Tue, 24 May 2011 09:30:19 +0530

I agree with Paul & Charles for using option 2 because option 1 is a the
subset of option 2.

Adding to their point, I'll provide my understanding of Media stream
object as follows:

Each Media stream object has metadata information about one recorded RTP
stream and its related media specific information. For the sake easy
understanding, I'm writing Media stream object in terms of format. Media
stream object shall be visualized in the format as follows: 

1) Start & Stop time attribute for the specific media stream. This will
be represented as Metadata XML attribute
2) Media reference ("label") attribute has the linkage between RS SDP
m-line and Media stream element in XML
3) Codec parameters shall refer to RS SDP codec specific attributes.

In the current model, each metadata object shall be mapped to one of the
m-line in RS SDP. How RS SDP m-line is mapped to CS-SDP m-line based on
SRC deployment. 

General MS object mechanism:
1) Each MS shall be contributed by multiple participant or single
participant. 
2) Each MS lifetime related to CSG because participant may move from one
CS to another CS within single CSG and also CSG will span across
multiple RS. 

Please read inline for more comments.

Thanks
Partha 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Paul Kyzivat (pkyzivat)
Sent: Tuesday, May 24, 2011 2:28 AM
To: Charles Eckel (eckelcu)
Cc: [email-masked]
Subject: Re: [siprec] What does an MS object represent? Can we pick one
of theseoptions?

	Thanks,
	Paul

On 5/23/2011 2:02 PM, Charles Eckel (eckelcu) wrote:
> break out the media on a per participant basis, it is more intuitive, 
> in my opinion, to go with option 2.
>
> Cheers,
> Charles
>
>

>>>> not everyone considers it this way, and I am prepared to be
<Partha> This may be achieved in some scenario but not always because
multiple contributing participant within single RTP stream is not
possible for SRC which are acting as RTP translator. </Partha>

>>>> not sufficiently descriptive in this case, but I can't come up with

<Partha> Even though both MS object representing audio, Metadata
information like MS helps to understand the real relationship with
participant & CS </Partha>

---

From: "Elwell, John" <[email-masked]>
Date: Tue, 24 May 2011 09:35:07 +0200

So those who have responded prefer option 2, and therefore I can agree to that. Essentially a Media Stream object models a recorded media stream. This should be made clear in the description of this object class in the model draft.

A recorded media stream can be contributed to by one, several or all Participants. If there are two recorded media streams relating to the same medium, there should be a means within the model to tie the two together. For example, in a 2-party audio call where there is one recorded media stream per direction, the two should somehow be indicated as related (e.g, by having a common value of a "content" attribute).

I am not sure I agree with Partha's further point about CSG. I will reply separately on that.

John (as individual)

> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Parthasarathi R (partr)
> Sent: 24 May 2011 05:00
> To: Paul Kyzivat (pkyzivat); Charles Eckel (eckelcu)
> Cc: [email-masked]
> Subject: Re: [siprec] What does an MS object represent? Can 
> we pick one of theseoptions?
> 
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf
> >>>> Subject: [siprec] What does an MS object represent? Can 
> we pick one
> > of
> >>> theseoptions?
> >>>>
> >>>> not sufficiently descriptive in this case, but I can't 
> come up with
> > a
> >>> better name.
>

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 24 May 2011 08:57:03 -0400

> So those who have responded prefer option 2, and therefore I can agree to that. Essentially a Media Stream object models a recorded media stream. This should be made clear in the description of thi…

Fair enough

> A recorded media stream can be contributed to by one, several or all Participants. If there are two recorded media streams relating to the same medium, there should be a means within the model to t…

While your example is a place where tying is perhaps important, I don't 
think the criteria for doing so is that they are the "same medium", at 
least not the way I think of medium.

AFAIK two streams would be considered "the same medium" if they are both 
video, or both voice. But not all such things should be tied together, 
at least not in the same way.

The matched pair of streams, one in each direction, between a pair of 
participants, that carries the two sides of a conversation, is a special 
relationship. And I agree that we currently seem to lack a way to 
represent that.

We could also have multiple streams of the same medium in a CS - for 
instance carrying multiple streams of video. That is a different 
relationship.

It could be a lot of work to figure out all the relationships that might 
need to be represented. I would suggest leaving most of that out of 
scope for now to avoid a rat hole. But we probably do need to describe 
the pairing of the two sides of a single conversation.

	Thanks,
	Paul

>> I agree with Paul&  Charles for using option 2 because option
>> 1 is a the
>> subset of option 2.
>>
>> 1) Start&  Stop time attribute for the specific media stream.
>> This will
>> possible for SRC which are acting as RTP translator.</Partha>
>>
>> participant&  CS</Partha>

---

From: "Elwell, John" <[email-masked]>
Date: Tue, 24 May 2011 15:17:34 +0200

> From: Paul Kyzivat [mailto:[email-masked]] 
> Sent: 24 May 2011 13:57
> To: Elwell, John
> Cc: Parthasarathi R (partr); Charles Eckel (eckelcu); [email-masked]
> > So those who have responded prefer option 2, and therefore 
> I can agree to that. Essentially a Media Stream object models 
> a recorded media stream. This should be made clear in the 
> description of this object class in the model draft.
> 
> Fair enough
> 
> > A recorded media stream can be contributed to by one, 
> several or all Participants. If there are two recorded media 
> streams relating to the same medium, there should be a means 
> within the model to tie the two together. For example, in a 
> 2-party audio call where there is one recorded media stream 
> per direction, the two should somehow be indicated as related 
> (e.g, by having a common value of a "content" attribute).
> 
[JRE] Yes and no. I had considered things like main video, presentation video, current speaker video as being different media (albeit of the same type, video). So we need some means of tying together the two or more sides of a single conversation, whether this be an audio conversation, a main video conversation, or whatever.

John (as individual)

> [mailto:[email-masked]] On
> >>>>> Behalf Of Elwell, John
> >>>>>> Where different participants are contributing to the 
> same medium,
> >>> this
> >>>>>> objects. For example, in a 2-party audio-only call, 
> there are two
> >>> MS

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 24 May 2011 10:56:17 -0400

Partha/Ram,

Can you add something to the metadata to tie together the two MS that 
represent the two sides of a single conversation? (m-lines with same 
offset in offer and answer of a CS dialog). I guess this is a new 
relationship in the metamodel. (Somehow I think it was in there once.)

	Thanks,
	Paul

---

From: "Ram Mohan R (rmohanr)" <[email-masked]>
Date: Tue, 24 May 2011 22:22:18 +0530

Paul/ John,

I am ok to add attribute "content" in MS block of the model. 

The question is do we want to map this attribute in the model to 
1) The Session Description Protocol (SDP) Content Attribute defined in
RFC 4796 OR 
2)define our own content in the metadata XML which can take values as
"audio" / "main video" / "presentation video" / "current speaker video"
as suggested by John.

Please let me know your inputs.

Ram

> Sent: Tuesday, May 24, 2011 8:26 PM
> To: Elwell, John
> Cc: [email-masked]
one
> > [JRE] Yes and no. I had considered things like main video,
> presentation video, current speaker video as being different media
> (albeit of the same type, video). So we need some means of tying
> together the two or more sides of a single conversation, whether this
> be an audio conversation, a main video conversation, or whatever.
> >
> > John (as individual)
> >
> >
> >>
> >> 	Thanks,
> >> 	Paul
> >>
> >>>>>> IIUC, option 1 can be achieved as one of the possibilities
> within
> >>>>> option 2.
> >>>>>>
> >>>>>> 	Thanks,
> >>>>>> 	Paul
> >>>>>>
> >>>>>>>> I think the fundamental question at issue is what exactly an
> MS
> >>>>> object
> >>>>>>> represents. Draft-00 is
> >>>>>>>> participants that are not separately identifiable and/or
where
> >>>>> their

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 24 May 2011 13:07:29 -0400

> "audio" / "main video" / "presentation video" / "current speaker video"

Well...

ISTM that if there is already an a=content in the CS SDP, then we would 
want to report that in the metadata.

If there is a pair (one from offer, one from answer, same m-line offset) 
and there is *not* an a=content that pairs them, then we would want to 
report it anyway.

But if we do that we have a problem with names and conflicts. rfc4796 
doesn't define a lot of names, nor does it say whether corresponding 
streams in offer & answer must have the same names.

So I think we have something similar but not quite the same, and we need 
to define something new. It must permit identifying streams even though 
we know nothing of their purpose.

	Thanks,
	Paul

---

From: "Parthasarathi R (partr)" <[email-masked]>
Date: Tue, 24 May 2011 23:47:14 +0530

I agree with John that there is a need for relationship between MS for
grouping. Also, I'm of the same opinion as Paul that "content" attribute
may not serve this requirement. 

I had offline discussion with Ram, we think that new "media stream
content group" metadata object may solve this issue.

The half-baked idea is to

1) Create new optional "Media Stream Content Group" (MSCG) metadata
object 
2) MSCG is related to MS as [0..*]-[1..*] 
3) MSCG is relate to CS as [0..*]- [1..*]

Please let me know your opinion on the same.

Thanks
Partha

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Paul Kyzivat (pkyzivat)
Sent: Tuesday, May 24, 2011 10:37 PM
To: Ram Mohan R (rmohanr)
Cc: [email-masked]
Subject: Re: [siprec] What does an MS object represent? Can we pick one
of theseoptions?

> RFC 4796 OR 2)define our own content in the metadata XML which can 
> take values as "audio" / "main video" / "presentation video" / 
> "current speaker video"
> as suggested by John.

Well...

	Thanks,
	Paul

---

From: Leon Portman <[email-masked]>
Date: Tue, 24 May 2011 21:37:16 +0300

I am OK with the below

Leon

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Parthasarathi R (partr)
Sent: Tuesday, May 24, 2011 9:17 PM
To: Paul Kyzivat (pkyzivat); Ram Mohan R (rmohanr)
Cc: [email-masked]
Subject: Re: [siprec] What does an MS object represent? Can we pick one of theseoptions?

I agree with John that there is a need for relationship between MS for grouping. Also, I'm of the same opinion as Paul that "content" attribute may not serve this requirement. 

I had offline discussion with Ram, we think that new "media stream content group" metadata object may solve this issue.

1) Create new optional "Media Stream Content Group" (MSCG) metadata object
2) MSCG is related to MS as [0..*]-[1..*]
3) MSCG is relate to CS as [0..*]- [1..*]

Thanks
Partha

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Paul Kyzivat (pkyzivat)
Sent: Tuesday, May 24, 2011 10:37 PM
To: Ram Mohan R (rmohanr)
Cc: [email-masked]
Subject: Re: [siprec] What does an MS object represent? Can we pick one of theseoptions?

Well...

ISTM that if there is already an a=content in the CS SDP, then we would want to report that in the metadata.

If there is a pair (one from offer, one from answer, same m-line offset) and there is *not* an a=content that pairs them, then we would want to report it anyway.

But if we do that we have a problem with names and conflicts. rfc4796 doesn't define a lot of names, nor does it say whether corresponding streams in offer & answer must have the same names.

So I think we have something similar but not quite the same, and we need to define something new. It must permit identifying streams even though we know nothing of their purpose.

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 24 May 2011 17:21:42 -0400

> Please let me know your opinion on the same.

This will presumably be used *a lot*, so its best if its not cumbersome.

It could be simpler if its limited to describing pairs of streams that 
are "mated". Then they could simply refer to one another.

It is perhaps more complex if you consider a conference, where each 
participant has a "pair", where one side is what it sends towards the 
focus and the other side is what it receives from the focus. Then the 
relationship is more complex.

We also *perhaps* need a way to represent the a=content things in the 
CS. But that can be considered a separate problem.

	Thanks,
	Paul

> streams in offer&  answer must have the same names.
>

---

From: "Elwell, John" <[email-masked]>
Date: Wed, 25 May 2011 08:14:59 +0200

Given that SDP's a=content might not be rich enough, I think we need to define something new for metadata, independent of a=content. The SRC could choose to derive the metadata content attribute from the a=content attributes of the contributing CS streams (that is an SRC decision), but should not use the a=content label in the RS SDP.

John (as individual)

> Sent: 24 May 2011 18:07
> Cc: Elwell, John; [email-masked]

---

From: "Elwell, John" <[email-masked]>
Date: Wed, 25 May 2011 15:39:01 +0200

I don't see a need for yet another object class for this. I think a separate object for such a group is needed only if the group can have its own attributes, and I don't see a need for that. I had expected it to be just another attribute of the MS class. I am struggling for a name for the attribute, but perhaps just "content" would suffice. MS objects having the same value for this attribute would be implicitly grouped.

John (as individual)

> [mailto:[email-masked]] On Behalf Of Paul Kyzivat
> Sent: 24 May 2011 22:22
> To: Parthasarathi R (partr)
> Cc: [email-masked]