Subject: [AVT] Re: ULP multiplexing schemes

From: Magnus Westerlund <[email-masked]>
Date: Wed, 05 Nov 2003 14:52:04 +0100

Hi Adam,

I thought this mail being of such general discussion value that I CC the 
mailing list in my response, I hope you don't feel hurt.

Adam Li wrote:

> Hi Magnus,
> 
> I have a couple of questions about multiplexing. 
> 
> RFC 3550 says "For efficient protocol processing, the number of
> multiplexing points should be minimized, as described in the integrated
> layer processing design principle." It specifically recommends against
> multiplexing multiple stream of different media type in the same session
> based on PT and SSRC. (It does recognize multiplexing media of the same
> type with SSRC.) With this, shouldn't we discourage multiplexing, i.e.,
> putting the media and FEC streams (they should be regarded as different
> media?) in the same address/port and multiplexing them only based on PT
> or SSRC?

Yes, and no. In some sense one might consider it as another media, and 
thus send it in a separate session. Using separate sessions has the 
advantage when one likes to be able do disconnect the ULP media. On the 
other hand the ULP stream is the same media as it encodes over. IT 
shares the timestamp behaviour, delivery requirements, etc. If you look 
in the list of argument in RFC 3550 against multiplexing separate media 
into the a single session you see that they does not apply for this case:

    1. If, say, two audio streams shared the same RTP session and the
       same SSRC value, and one were to change encodings and thus acquire
       a different RTP payload type, there would be no general way of
       identifying which stream had changed encodings.

This does not apply as they after reconstruction are a single stream.

    2. An SSRC is defined to identify a single timing and sequence number
       space.  Interleaving multiple payload types would require
       different timing spaces if the media clock rates differ and would
       require different sequence number spaces to tell which payload
       type suffered packet loss.

As the ULP simply integrates into the timestamp space and does not 
really use it, there is no problem. The sequence number does for most 
payload formats not create any problem, the current exceptions are some 
fragmented video formats and RFC 2793. It is after all if the there is 
needed to know if the lost packet was an ULP packet or real media that 
enables the usage of this or not.

    3. The RTCP sender and receiver reports (see Section 6.4) can only
       describe one timing and sequence number space per SSRC and do not
       carry a payload type field.

This is not a concern for ULP as it does not use the TS space, and 
packet loss reports can just as well be done aggregated then separated.

    4. An RTP mixer would not be able to combine interleaved streams of
       incompatible media into one stream.

An mixer is an user of ULP to recovery the stream for mixing therefore 
it is not a problem.

    5. Carrying multiple media in one RTP session precludes: the use of
       different network paths or network resource allocations if
       appropriate; reception of a subset of the media if desired, for
       example just audio if video would exceed the available bandwidth;
       and receiver implementations that use separate processes for the
       different media, whereas using separate RTP sessions permits
       either single- or multiple-process implementations.

This is a valid reason and in cases where it applies the session 
multiplexing should be used.

    Using a different SSRC for each medium but sending them in the same
    RTP session would avoid the first three problems but not the last
    two.

So in summary, there are two of these points that applies for ULP as I 
see it: 2 and 5. And this is fully dependent on the use cases and media 
transported. I think the concerns around point 2 is the more important 
one and these can be addressed though the usage of RFC 2198 for ULP 
information.

> 
> Another question is about the Session Multiplexing. It seems that we
> usually refer to multiplexing as the practice of putting multiple
> streams in the same socket connection (same address/port pair), and
> separate the streams based on some marks on the packets. If two streams
> are in separate sessions, then maybe it is a little confusing to call
> them multiplexing. 

Yes, however there is a clear binding between the ULP stream and its 
original data stream. Without the capability to bind them together the 
ULP information becomes useless.

> 
> I guess you are more concerned about how we can associate the FEC
> streams to the payload streams under different scenarios, right? Sending
> the FEC stream in another session seems is just a very special case of
> the many other ways to send the FEC stream separately. And it is a quite
> un-natural way. It is not obvious how to associate a stream in one
> session to another stream in another session? I feel it may not be
> necessary to specify all the details for every combinations and
> possibilities, because there are simply way too many of them to send two
> streams separately. 

This is exactly my concerns, and I do not agree that there is not need 
to specify how they are used. I can agree that not all possible needs to 
be specified, however I feel that the different major cases needs to be 
defined.

As I see it we have three different ways that for different usage are 
needed due to different scenarios, these should be clearly specified how 
to use. The three cases are:

- PT multiplexing, restricted for some payload formats that needs it 
sequence number untouched, RFC 2793 and some fragmentation schemes in 
some payload formats, like H.264 will not like it.

- Usage of RFC 2198 to include ULP data as redundant data.

- Sending ULP in separate RTP session, need when one wants to have 
separate treatment of the data streams or avoid sending useless data to 
receivers that can't decode it.

> 
> My suggestion is that we simply mention in the draft that PT
> multiplexing is possible, SSRC multiplex is not possible, and refer to
> the RFC 3550 to discourage multiplexing FEC stream with the payload
> through the same address/port. 
> 

All of the three above mentioned methods has separate concerns regarding 
signalling and usage. For example the session multiplexing scheme needs 
to use grouping of media lines (RFC3388) to bind them together 
explicitly.  There are also SSRC consideration rules for this, see the 
RTP retransmission draft. The usage of RED is already in there, and some 
paragraphs about the PT multiplexing and its concerns are also needed. I 
would also include a motivation why SSRC multiplexing is not a good method.

Cheers

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt