Subject: [AVTCORE] Comments on draft-begen-mmusic-redundancy-grouping-01

From: Magnus Westerlund <[email-masked]>
Date: Fri, 22 Jul 2011 14:39:00 +0200

Hi,

(Primarily as individual, but the WG home question is as WG chair)

I have reviewed this draft and some comments on it.

1. If this solution proposal is to be taken forward I think it needs to
be discussed in AVTCORE. The reason is that it has several RTP details
that needs to be figured out to make stream duplication issue free in a
general usage. I will take up a few of these issues in the below comments.

2. One aspect this document has is a relation to what is discussed in
draft-westerlund-avtcore-multistream-and-simulcast-00. Stream
duplication both results in a certain form of multi-stream and also has
a similar but slightly different semantics to simulcast in that the
streams are two representations of a single original media source. In
this case they are clones, but not true duplicates due to that RTP
doesn't handle try duplicates well.

3. Another issue that also the authors has identified is the question of
what one do when one has multiple media streams from the same end-point.
And in a general case one can't use the CNAME to separate the sources.
The reason is that two media sources that has a common timing
(synchronization) context do needs to indicate that by having the same
CNAME. Thus there are need for some additional mechanism to ensure that
SSRC carrying RTP media streams that are duplicates of each other do in
fact be labeled explicitly. I also note that a SDP based solution is not
sufficient to resolve this issue. I think we need something in RTP, so
that SSRC collisions can be handled.

4. In addition there is a few questions how this duplication really
should set some RTP fields. Is the duplication only working on Sequence
number level? There seem to be additional questions, for example in the
need to have RTP timestamp considerations between the streams. They need
to point identically into the joint timeline. Otherwise outages, or
receiver taking synch from one and not the other stream can end up with
different synchronizations. Becomes especially interesting to discuss
when the time shifted transmission.

5. The draft also discussed Payload Type differences and have them in
the SDP examples. I don't see how that is appropriate. If the streams
are true encoding duplicates then they should use the same payload type
configuration, and thus a single payload type should be sufficient and
in fact recommended, unless the source encoding truly changes.

6. I think it is worth discussing also the benefits of SSRC multiplexed
versus the session multiplexed. In multicast context there is clear
benefits from a control and congestion perspective to have session
multiplexed versions and these mapped onto different multicast groups
enabling a receiver to choose one or both streams.

7. Thus I think the signalling aspects of this is the minor part.
Although there clearly are some aspects of this needing discussion. For
example clearer offer/answer rules for handling these.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------

---

From: "Ali C. Begen (abegen)" <[email-masked]>
Date: Sun, 24 Jul 2011 20:21:58 -0700

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Magnus Westerlund
> Sent: Friday, July 22, 2011 8:39 AM
> To: mmusic (E-mail); IETF AVTCore WG
> Subject: [AVTCORE] Comments on draft-begen-mmusic-redundancy-grouping-01
> 
> Hi,
> 

Thanks for the review.
 
> general usage. I will take up a few of these issues in the below comments.

That is why we asked for a slot in avtcore on wed.
 
> doesn't handle try duplicates well.

The data carried in the redundant streams are identical but the rtp header differs due to the different SSRCs. That is all. SSRC muxing nicely fits the use case we are tackling with. Simulcasting is quite different from our use case because in simulcasting different RTP streams most likely carry different data.

>From the same end-point (e.g., source) there could of course of multiple media streams. You can use SSRC muxing for example to identify your streams and use ssrc-group to group them if needed.

The concern in that editorial note was what if there were multiple streams in each m line but what if you wanted grouping/association among only a subset.

For example,

        m=video 30000 RTP/AVP 101 102
        ....
        m=video 30000 RTP/AVP 103 104

I can easily associate 101 and 102 if I want to, but what if 101 and 103 were associated with each other. Can I use a=group line? Probably not. But, I don't know why you would need that kind of relation in the first place (even if you need, there are probably less messy ways to express them). If that had been a very desirable feature, SDP and grouping semantics along with RTP would not have defined in their current form I guess.

> CNAME. Thus there are need for some additional mechanism to ensure that

If 101 and 103 are associated, why can't I use the same CNAME for those streams? If 102 and 104 are associated/related, I would use a common CNAME for them, too (but theirs would be separate from 101 and 103's).

> that SSRC collisions can be handled.

I can't say I am following.

> should set some RTP fields. Is the duplication only working on Sequence

That is the easiest way for a middle box or endpoint doing the stream merging (duplicate suppression). If seqnums are different, what would one do? look into the payload?

> when the time shifted transmission.

The duplicate streams are coming from the same source, so they have the same timestamp as well as seqnums. It has to be this way because the node merging the streams will not re-write anything at all. It is a simple suppression operation. Beauty of it is that things stay pretty simple to use and implement without any changes to RTP/RTCP plane at all.

As for the time-shifted transmission, I suppose you are referring to the temporal diversity/delayed duplication (draft-begen-mmusic-temporal-interleaving-02). Well, in that case, timestamps are still the same for the reasons above. Changing the timestamps for the delayed packets does not bring any benefit (actually it complicates things) since the ultimate RTP application will still receive one and only one RTP stream.

> the SDP examples. I don't see how that is appropriate. If the streams

When you are doing SSRC muxing, you put multiple streams in the same m line and they end up having different PT numbers. What is wrong with that? If they are in separate m lines, they could have the same PT number.

> in fact recommended, unless the source encoding truly changes.

See above.

> enabling a receiver to choose one or both streams.

This is what we have in section 5.2. Different streams can be sent in different multicast sessions but it does not mean that they are separate RTP sessions. I discussed this in detail with Colin. Thanks to him for trying to teach me over and over what an RTP session actually means.
 
> example clearer offer/answer rules for handling these.

We have some text around O/A stuff. But frankly, I don’t see any use case where duplication will be used in a unicast application. I mean if redundancy is needed, one should use FEC instead for unicast transport. Duplication is a nice use case for multicast transport where the bw overhead is easier to justify and the implementation is easier. And one would naturally implement this spec in networks where he already knows that the network elements support this spec. So, maybe we should remove the whole O/A text anyway and write that this will only be used declaratively.

-acbegen
 
> _______________________________________________
> Audio/Video Transport Core Maintenance
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 25 Jul 2011 06:54:10 -0400

On 2011-07-24 23:21, Ali C. Begen (abegen) wrote:
> 
> 
>> -----Original Message----- From: [email-masked]
>> [mailto:[email-masked]] On Behalf Of Magnus Westerlund Sent:
>> Friday, July 22, 2011 8:39 AM To: mmusic (E-mail); IETF AVTCore WG 
>> Subject: [AVTCORE] Comments on
>> draft-begen-mmusic-redundancy-grouping-01
>> 
>> Hi,
>> 
>> 1. If this solution proposal is to be taken forward I think it
>> needs to be discussed in AVTCORE. The reason is that it has several
>> RTP details that needs to be figured out to make stream duplication
>> issue free in a general usage. I will take up a few of these issues
>> in the below comments.
> 
>> 2. One aspect this document has is a relation to what is discussed
>> in draft-westerlund-avtcore-multistream-and-simulcast-00. Stream 
>> duplication both results in a certain form of multi-stream and also
>> has a similar but slightly different semantics to simulcast in that
>> the streams are two representations of a single original media
>> source. In this case they are clones, but not true duplicates due
>> to that RTP doesn't handle try duplicates well.
> 
> The data carried in the redundant streams are identical but the rtp
> header differs due to the different SSRCs. That is all. SSRC muxing
> nicely fits the use case we are tackling with. Simulcasting is quite
> different from our use case because in simulcasting different RTP
> streams most likely carry different data.

I agree that the purpose for multiple RTP streams from the same original
media source are different from the primary case I set out to solve. But
as Jonathan Lennox pointed out, redundancy and FEC have very similar
behaviors to simulcast with different encodings of the same original
source.

Thus, I do see a need to discuss for example how you identify the
streams that are a duplicate of each other.

> 
>> 3. Another issue that also the authors has identified is the
>> question of what one do when one has multiple media streams from
>> the same end-point.
> 
>> From the same end-point (e.g., source) there could of course of
>> multiple media streams. You can use SSRC muxing for example to
>> identify your streams and use ssrc-group to group them if needed.
> 
> The concern in that editorial note was what if there were multiple
> streams in each m line but what if you wanted grouping/association
> among only a subset.
> 
> For example,
> 
> m=video 30000 RTP/AVP 101 102 .... m=video 30000 RTP/AVP 103 104
> 
> I can easily associate 101 and 102 if I want to, but what if 101 and
> 103 were associated with each other. Can I use a=group line? Probably
> not. But, I don't know why you would need that kind of relation in
> the first place (even if you need, there are probably less messy ways
> to express them). If that had been a very desirable feature, SDP and
> grouping semantics along with RTP would not have defined in their
> current form I guess.

Yes, that is a very valid question also for simulcast.

And I think we can provide a story for doing that. But I don't think it
should be based on using the RTP payload type to identify either of the
versions.

> 
>> And in a general case one can't use the CNAME to separate the
>> sources. The reason is that two media sources that has a common
>> timing (synchronization) context do needs to indicate that by
>> having the same CNAME. Thus there are need for some additional
>> mechanism to ensure that
> 
> If 101 and 103 are associated, why can't I use the same CNAME for
> those streams? If 102 and 104 are associated/related, I would use a
> common CNAME for them, too (but theirs would be separate from 101 and
> 103's).

If SSRC A has a duplciation in SSRC C and SSRC B has a duplication in
SSRC D and both A and B comes from the same end-point and have a common
synchronization context, then all of A-D needs to have the same CNAME.

And I get the impression that you are using a payload type based
association scheme here. And I think that is a bad idea. For example it
can't support particular many streams. It also looks semantically wrong,
as if Stream A uses PT 96, then SSRC C should use PT 96 also for the
corresponding packet duplicates. And if A switches from one PT to
another the other stream needs to follow.

Thus I think a SDES item based approach for indicating that it is in
fact A and C that are duplicates, rather than A and B or A and D.

> 
>> SSRC carrying RTP media streams that are duplicates of each other
>> do in fact be labeled explicitly. I also note that a SDP based
>> solution is not sufficient to resolve this issue. I think we need
>> something in RTP, so that SSRC collisions can be handled.
> 

I suggest that you use something like the SRCNAME SDES item extension I
propose in my draft.

> 
>> 4. In addition there is a few questions how this duplication
>> really should set some RTP fields. Is the duplication only working
>> on Sequence
> 
> That is the easiest way for a middle box or endpoint doing the stream
> merging (duplicate suppression). If seqnums are different, what would
> one do? look into the payload?

It was simply that you had not documented what needs to be the same.
Sequence number being the same is likely the easiest, but one do have to
raise some warning flags if one has processing that changes these
streams. For example a translator modifying the packetization must do
the exact same manipulation on the other stream.

But, I don't think anything else really works.

> 
>> number level? There seem to be additional questions, for example in
>> the need to have RTP timestamp considerations between the streams.
>> They need to point identically into the joint timeline. Otherwise
>> outages, or receiver taking synch from one and not the other stream
>> can end up with different synchronizations. Becomes especially
>> interesting to discuss when the time shifted transmission.
> 
> The duplicate streams are coming from the same source, so they have
> the same timestamp as well as seqnums. It has to be this way because
> the node merging the streams will not re-write anything at all. It is
> a simple suppression operation. Beauty of it is that things stay
> pretty simple to use and implement without any changes to RTP/RTCP
> plane at all.

Yes, but you hadn't made it clear, and you have both SSRC and implied
change of PT. So I didn't want to assume anything.

The timestamp field also have implcations due to its binding with RTCP
SR information.

> 
> As for the time-shifted transmission, I suppose you are referring to
> the temporal diversity/delayed duplication
> (draft-begen-mmusic-temporal-interleaving-02). Well, in that case,
> timestamps are still the same for the reasons above. Changing the
> timestamps for the delayed packets does not bring any benefit
> (actually it complicates things) since the ultimate RTP application
> will still receive one and only one RTP stream.

Ok, but you will need to make clear on how one should set the RTCP SR
timestamp NTP mapping to be clear. And what that implies for the RTCP
jitter calculation.

> 
>> 5. The draft also discussed Payload Type differences and have them
>> in the SDP examples. I don't see how that is appropriate. If the
>> streams
> 
> When you are doing SSRC muxing, you put multiple streams in the same
> m line and they end up having different PT numbers. What is wrong
> with that? If they are in separate m lines, they could have the same
> PT number.

I am still asking why, there is no reason to use different PTs, unless
you are doing the association between the different streams by them
having different PTs. Which I believe is a Bad Idea (TM). The encoding
in the packet are the same, so why don't they have the same PT.

> 
>> 6. I think it is worth discussing also the benefits of SSRC
>> multiplexed versus the session multiplexed. In multicast context
>> there is clear benefits from a control and congestion perspective
>> to have session multiplexed versions and these mapped onto
>> different multicast groups enabling a receiver to choose one or
>> both streams.
> 
> This is what we have in section 5.2. Different streams can be sent in
> different multicast sessions but it does not mean that they are
> separate RTP sessions. I discussed this in detail with Colin. Thanks
> to him for trying to teach me over and over what an RTP session
> actually means.

Yes, you have an example, but not a discussion of why an implementor
should choose Session multiplexing over SSRC. I think that should be
included, or I think you can wait for the architectural document I think
will result from my multi-stream and simulcast draft.

>> Although there clearly are some aspects of this needing discussion.
>> For example clearer offer/answer rules for handling these.
> 
> We have some text around O/A stuff. But frankly, I don’t see any use
> case where duplication will be used in a unicast application. I mean
> if redundancy is needed, one should use FEC instead for unicast
> transport. Duplication is a nice use case for multicast transport
> where the bw overhead is easier to justify and the implementation is
> easier. And one would naturally implement this spec in networks where
> he already knows that the network elements support this spec. So,
> maybe we should remove the whole O/A text anyway and write that this
> will only be used declaratively.

Then, I think we should slap an application statement to the beginning
of the draft to ensure that it may only be used in multicast. For
example b=TIAS was only intended to used in declarative SDPs, but it has
become common in O/A, but it doesn't have clear rules.

So I think it imporant that one don't assume to much. Yes, it might not
be the best idea for unicast, but that is not going to prevent it from
being used.

Cheers

Magnus Westerlund

---

From: David R Oran <[email-masked]>
Date: Mon, 25 Jul 2011 09:38:39 -0400

One small comment, basically agreeing with Magnus point about payload types.

Usually when people use payload identifiers to do demultiplexing they later regret it. Payload identifiers (in RTP or any other protocol for that matter) should only be used to decide which parser to hand the data to, not which state variable to bind the data in the packet to. 

(further aside: this is one of the few things the OSI guys actually got right when they had both payload ids and SAPs at every layer).

DaveO.

---

From: "Ali C. Begen (abegen)" <[email-masked]>
Date: Mon, 25 Jul 2011 07:07:34 -0700

> From: Magnus Westerlund [mailto:[email-masked]]
> Sent: Monday, July 25, 2011 6:54 AM
> To: Ali C. Begen (abegen)
> Cc: mmusic (E-mail); IETF AVTCore WG
> Subject: Re: [AVTCORE] Comments on draft-begen-mmusic-redundancy-grouping-01
> 

Ok, lets discuss it. My current proposal is to use different SSRCs. PTs turn out to be different since I designate these streams in the same m line. Unless I am mistakes, you cannot put two different streams with the same PT in a single m line. Remember, we do not use PT to demux the RTP streams, we use SSRC for that purpose.
 
> synchronization context, then all of A-D needs to have the same CNAME.

Well, above implies that all streams are coming from the same source and yes, they will end up having the same CNAME since you required A will be synched with B. But, still I think this is workable without any additional mechanism. A and C would be in the same m line and B and D would be in the same (but separate from the first) m line. That would give me what I want.
 
> another the other stream needs to follow.

No, I am not suggesting to use PT for demuxing. I use SSRC for that purpose. As I asked above, if I can use the same PT for multiple streams in the same m line, then I will. But I don’t think I can.

> It was simply that you had not documented what needs to be the same.

I think I did, but I will check.

> the exact same manipulation on the other stream.

Yes, obviously. We can put this warning.
 
> But, I don't think anything else really works.

Right, seqnum is the easiest and possibly the only way in practice.
 
> change of PT. So I didn't want to assume anything.

OK, will make this clear, as well.
 
> timestamp NTP mapping to be clear. And what that implies for the RTCP

The fact that you are using delayed duplication or spatial diversity means that you will likely introduce some jitter into your stream unless the node merging the streams smooths out that jitter in the first place. There is not nothing special for the ultimate endpoint for preparing the reports, it just does  what it is supposed to do. In most systems, the ultimate RTP receivers might not be even aware of the fact that they are receiving a stream merged from two identical streams. That is a design feature.

> in the packet are the same, so why don't they have the same PT.

Well again, I am not using it for demuxing purposes. I just believe that is how the m line works. Tell me otherwise and I will be happy to make PTs the same.
 
> will result from my multi-stream and simulcast draft.

Lets discuss in the session.
 
> of the draft to ensure that it may only be used in multicast. For

Wfm.

> be the best idea for unicast, but that is not going to prevent it from

If one really uses this for unicast, let them deal with all the potential O/A problems that might surface. It is more like they will deserve it :)

Cheers,
-acbegen

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 25 Jul 2011 10:55:33 -0400

On 2011-07-25 10:07, Ali C. Begen (abegen) wrote:

> Ok, lets discuss it. My current proposal is to use different SSRCs.
PTs turn out to be different since I designate these streams in the same
m line. Unless I am mistakes, you cannot put two different streams with
the same PT in a single m line. Remember, we do not use PT to demux the
RTP streams, we use SSRC for that purpose.

No, that is wrong. The PT is completely unrelated to to which SSRC that
use them. The set of configured PTs can be used by any of the sources
that transmit in the direction for which they are configured. I do
include the direction because of the O/A aspects of having different
configurations for receving in different directions.

>> If SSRC A has a duplciation in SSRC C and SSRC B has a duplication
>> in SSRC D and both A and B comes from the same end-point and have a
>> common synchronization context, then all of A-D needs to have the
>> same CNAME.
> 
> Well, above implies that all streams are coming from the same source
> and yes, they will end up having the same CNAME since you required A
> will be synched with B. But, still I think this is workable without
> any additional mechanism. A and C would be in the same m line and B
> and D would be in the same (but separate from the first) m line. That
> would give me what I want.

I am not agreeing that you would put A and B in different sessions. This
doesn't make sense if you actually like to provide two related media
streams of the same type, for example two cameras covering different
parts of a telepresence room, or two different camera angels in a movie.

What you are suggesting seems to be a hack to just get around the issue
that if you have streams that are some form of alternatives of the
actual same original source then you need to identify, like the SRCNAME
proposal.

>> association scheme here. And I think that is a bad idea. For
>> example it can't support particular many streams. It also looks
>> semantically wrong, as if Stream A uses PT 96, then SSRC C should
>> use PT 96 also for the corresponding packet duplicates. And if A
>> switches from one PT to another the other stream needs to follow.
> 
> No, I am not suggesting to use PT for demuxing. I use SSRC for that
> purpose. As I asked above, if I can use the same PT for multiple
> streams in the same m line, then I will. But I don’t think I can.
> 

Yes, understand this is a different type of missunderstanding of RTP.

>> 
>> Ok, but you will need to make clear on how one should set the RTCP
>> SR timestamp NTP mapping to be clear. And what that implies for the
>> RTCP jitter calculation.
> 
> The fact that you are using delayed duplication or spatial diversity
> means that you will likely introduce some jitter into your stream
> unless the node merging the streams smooths out that jitter in the
> first place. There is not nothing special for the ultimate endpoint
> for preparing the reports, it just does  what it is supposed to do.
> In most systems, the ultimate RTP receivers might not be even aware
> of the fact that they are receiving a stream merged from two
> identical streams. That is a design feature.
> 

When you talk about jitter here, you are talking about small transport
level jitter, rather than jitter on the level of frame intervals.

Sure, if you have a translator that merge these, then you clearly should
be hidden to the end-point behind translator.

>>>> 7. Thus I think the signalling aspects of this is the minor
>>>> part. Although there clearly are some aspects of this needing
>>>> discussion. For example clearer offer/answer rules for handling
>>>> these.
>>> 
>>> We have some text around O/A stuff. But frankly, I don’t see any
>>> use case where duplication will be used in a unicast application.
>>> I mean if redundancy is needed, one should use FEC instead for
>>> unicast transport. Duplication is a nice use case for multicast
>>> transport where the bw overhead is easier to justify and the
>>> implementation is easier. And one would naturally implement this
>>> spec in networks where he already knows that the network elements
>>> support this spec. So, maybe we should remove the whole O/A text
>>> anyway and write that this will only be used declaratively.
>> 
>> Then, I think we should slap an application statement to the
>> beginning of the draft to ensure that it may only be used in
>> multicast. For
> 
> Wfm.
> 
>> example b=TIAS was only intended to used in declarative SDPs, but
>> it has become common in O/A, but it doesn't have clear rules.
>> 
>> So I think it imporant that one don't assume to much. Yes, it might
>> not be the best idea for unicast, but that is not going to prevent
>> it from being used.
> 
> If one really uses this for unicast, let them deal with all the
> potential O/A problems that might surface. It is more like they will
> deserve it :)
> 

:-)

Cheers

Magnus Westerlund