Subject: [AVT] Multiplexing streams with different resolutions in one RTP Session

From: "Franceschini Guido" <[email-masked]>
Date: Tue, 2 May 2006 14:54:20 +0200

Dear all,

a few months ago I already posted some mail on this issue (see the thread on "Issue with different timestamp frequencies", and got some feedback.
Now I have progressed with my implementation and would like to share with you my concerns/solutions.

The problem I am talking about is: how to multiplex in a single RTP Session streams with different time resolutions.

There are two scenarios -at least- where this occurs:
a) AMR-WB (16KHz) + DTMF (8KHz)
b) AMR-WB (16KHz) + another codec narrowband (8KHz)

I have explored several approaches:

1) use the current specs, following the explaination from Magnus (mail of 31/01/2006)
PROS:
- no spec change (although IMHO the explaination from Magnus can be hardly derived from the text in the specs)
CONS:
- a complex receiver is required to identify the instants at which the time resolution has switched and so reconstruct the correct timeline
- in case a packet loss occurs near to the time resolution switch, an ambiguity in the timestamp of the packet cannot be resolved until the next RTCP report. If multiple such events occur between two RTCP reports, the ambiguity cannot be solved not even a posteriori.
- even if no time resolution switch occurs, concurrence of packet losses and audio suppression might induce the same ambiguity as before, since this case cannot be univocally distinguished anymore.
- if the time resolution switch occurs right at the time an RTCP report is produced, it is unclear which resolution shall be used within that RTCP report
- RTCP jitter calculation is unclear

2) use separate RTP sessions, i.e. separate destination addresses.
PROS:
- easy
- no ambiguity
CONS:
- 2 m=audio lines are required in SDP
- more network resources involved
- more NAT related problems
- need RTCP to sync the two flows

3) use a new MIME (X-DTMF) with 16 KHz resolution.
PROS:
- easy
- no ambiguity
CONS:
- extra SDP length
- does not address scenario b)

4) use separate SSRC for each time resolution
PROS:
- no ambiguity
CONS:
- need RTCP to sync the two flows
- RTCP rules might need some clarification

As you can guess from the list above, my favoured solution is #4. To complete it, however, I ask your help in interpreting the specs with reference to RTCP.
The 'root' of my concerns is that RTCP SR and RR require that one SSRC value is specified to indicate the sender. Unfortunately I have 2 SSRC values now...
Usage of 2 SSRC for sending within a single RTP Session is explicitely allowed (or even requested) in RFC3550, page 9 (look at the last sentence):

   Synchronization source (SSRC): The source of a stream of RTP
      packets, identified by a 32-bit numeric SSRC identifier carried in
      the RTP header so as not to be dependent upon the network address.
      All packets from a synchronization source form part of the same
      timing and sequence number space, so a receiver groups packets by
      synchronization source for playback.  Examples of synchronization
      sources include the sender of a stream of packets derived from a
      signal source such as a microphone or a camera, or an RTP mixer
      (see below).  A synchronization source may change its data format,
      e.g., audio encoding, over time.  The SSRC identifier is a
      randomly chosen value meant to be globally unique within a
      particular RTP session (see Section 8).  A participant need not
      use the same SSRC identifier for all the RTP sessions in a
      multimedia session; the binding of the SSRC identifiers is
      provided through RTCP (see Section 6.5.1).  If a participant
      generates multiple streams in one RTP session, for example from
      separate video cameras, each MUST be identified as a different
      SSRC.

RFC3550 however does not help much when addressing RTCP. On page 34 I found:

   The SR is issued if a
   site has sent any data packets during the interval since issuing the
   last report or the previous one, otherwise the RR is issued.

   Both the SR and RR forms include zero or more reception report
   blocks, one for each of the synchronization sources from which this
   receiver has received RTP data packets since the last report.

My interpretation is that the above text also implies:

   In case multiple SSRC are used by a site for sending, the SR for SSRCx
   is issued if a site has sent any data packets with that SSRCx during
   the interval since issuing the last report for SSRCx or the previous
   one; if no SR is needed, for any of the SSRCx in use for sending, then
   the RR is issued, indicating as sender's SSRC any of the SSRCx in use
   for sending.
   In case data packets have been sent with more than one SSRC, since SR
   only allow for describing the traffic generated by a single SSRC, only
   one SSRC can be elected to be represented in a SR report. This makes
   RTCP management quite difficult. An alternative idea is to combine
   multiple SR into a compound RTCP packet: in such a case the first SR
   will include a 'true' RR, while the additional SR will include only
   empty RR: this technique does not seem to contradict any rule, in
   particular for RR, since no data is received between the first and the
   subsequent SR in the compound RTCP packet!

Whatever the solution selected, I believe that some extra wording in the RFC3550 would be needed, in order to either clarify the use case or set new constraints (e.g.: that media with different time resolution cannot share the same SSRC within an RTP Session).
For example for solution 1), the specs do not really describe how to perform a time resolution switch at the sender side, nor how to identify it at a receiver: furthermore significant areas of ambiguity are introduced in such a case but totally ignored.

I welcome your comments

Thanks in advance
Guido Franceschini

--------------------------------------------------------------------

CONFIDENTIALITY NOTICE

This message and its attachments are addressed solely to the persons above and may contain confidential information. If you have received the message in error, be informed that any use of the content hereof is prohibited. Please return it immediately to the sender and delete the message. Should you have any questions, please contact us by replying to [email-masked].

        Thank you

                                        www.telecomitalia.it

--------------------------------------------------------------------

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: <[email-masked]>
Date: Wed, 3 May 2006 19:11:03 +0530

Hi Guido,
 
I have a simple and scalable proposition to different resolution of RTP timestamp. One can use the absolute time scale based on the sample rate. In SDP each payload type is associated with the sample rate and the same sample rate should be passed to lower layer voice processing. When payload type changes from AMR-WB to 8kHz DTMF the RTP timestamp should scale (or provide resolution) to 8kHz from that point onwards. As far as RTCP reports calculation is concerned we need to introduce absolute time as against the RTP timestamp. This should be scalable for any scenarios like codec switchover and other audio sample rates etc. Of course this may need to add the timescale/sample rate in RTCP calculation. In case of loss packets (in the event that we don't know what payload type is lost, timescale of previous payload type should be used.
 
Regards,
Rupesh

________________________________

From: Franceschini Guido [mailto:[email-masked]]
Sent: Tue 5/2/2006 8:54 AM
To: [email-masked]
Cc: Oldrini Stefano; Madaro Livio
Subject: [AVT] Multiplexing streams with different resolutions in one RTPSession

Dear all,

The SR is issued if a

I welcome your comments

Thanks in advance
Guido Franceschini

--------------------------------------------------------------------
CONFIDENTIALITY NOTICE
This message and its attachments are addressed solely to the persons
above and may contain confidential information. If you have received
the message in error, be informed that any use of the content hereof
is prohibited. Please return it immediately to the sender and delete
the message. Should you have any questions, please contact us by
replying to [email-masked] <mailto:[email-masked]> .
        Thank you
                                        www.telecomitalia.it <http://www.telecomitalia.it> 
--------------------------------------------------------------------
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 03 May 2006 17:13:11 +0200

Hi Guido,

A few comments inline.

> a) AMR-WB (16KHz) + DTMF (8KHz)

I still don't see why you keep on arguing for this case. To me this is a 
wrongly configured session. It should change the timestamp rate of the 
DTMF to 16kHz.

> b) AMR-WB (16KHz) + another codec narrowband (8KHz)

This is a reel one I agree. However it do simplify the issues of rapid 
switching which in my eyes should really not occur in this case.

> - does not address scenario b)

Not needed, the standard tones and telephone event payloads do support 
any rate.

>    subsequent SR in the compound RTCP packet!

In my opinion the right way of handling multiple SSRC at one end-point 
is to treat them as they are each individual end-points part of a 
multicast where everyone sees everyone. This implies that each SSRC will 
send its own SR when an active sender, otherwise sending RRs. The 
bandwidth distribution is done using the normal rules. Which will result 
in that for one end-point with two SSRC and another end-point with a 
single SSRC the first one will commonly use twice the RTCP bandwidth 
compared to the other out of the common pool.

If one include empty RR then one are in fact reporting that one don't 
see the other sender co-located at the same host. That implies an error 
and should those not be done.

> For example for solution 1), the specs do not really describe how to perform a time resolution switch at the sender side, nor how to identify it at a receiver: furthermore significant areas of ambi…

I agree that this is insufficiently described in RFC 3550. Are you 
willing to create a draft that tackles this issue?

Cheers

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]