Subject: [xrblock] Multiple channels in RTP streams

From: Alan Clark <[email-masked]>
Date: Wed, 16 Nov 2011 21:46:02 -0500

On the question of multiple channels in RTP streams - it may be helpful to
have some practical examples to use for reference and as examples.  This may
also answer one question in the XRBLOCK session which related to the
numbering of the channels in the case of multichannel audio.

(i) Multichannel Audio
RFC3551 defines an RTP profile for Audio and Video Conferences.  Section 4
describes the convention for multichannel sample based audio encodings -
which is to interleave samples using the orderings

Channels        1    2    3    4    5    6
2                      L    R
3                      L    R    C
4                      L    C    R    S
5                      FL  FR  FC  SL  SR
6                      L    LC   C   R    RC    S

This is generally consistent with the AIFF-C format, which is widely used
for multichannel audio - and uses this interleaved approach.  Note that this
does define an ordering, and hence channels 1, 2, 3, 4.. can be interpreted
in terms of Left, RIght etc.  Other IETF RFC¹s that relate to multichannel
audio include RFC3190, RFC4184, RFC5691 .....

(ii) MPEG Transport Streams
An MPEG Transport stream comprises multiple channels of video and/or audio
carried in MPEG Transport packets, with each media stream identified by PID.
A typical single program stream might have, for example, one video stream
and two audio streams (for two languages) plus additional streams for
captions, contents (PAT/PMT).  This MPEG Transport Stream is carried in a
single RTP session.  A typical use case would require the reporting of a
Video MOS for the video stream and an Audio MOS for each of the audio
streams.
 
(iii) Multi-view Video
Multi-view video generally covers 3D, multiple viewpoint video and other
similar applications.  Work on this has been performed within MPEG/VCEG as
extensions to H.264 and an RTP payload format was proposed in
draft-ietf-avt-rtp-mvc.

Best Regards

Alan

---

From: Qin Wu <[email-masked]>
Date: Tue, 22 Nov 2011 21:14:34 +0800

Multiple channels in RTP streamsIt looks multichannel in 3 cases has different meaning and not sure we can interpret or abstract it into the channel concept.
E.g., In multiple channel audio case(i.e., stereo audio application), multi-channel can be categorize into left audio channel, right audio channel, etc.
In Multiview video case, multiview can be catorized into left view, right view, etc.
I am not sure we can unify multiview video and multi-channel audio into channel concept.

In MPEG Transport Stream case, a transport stream used in digital television might contain multiple programs, to represent multiple television channels
each channel may consists of multiple video streams, multiple audio streams, and any necessary metadata, if we take 13 bits PID as channel ID or identity for each elementary stream(e.g.,
video stream, audio stream), channel ID is quite confusing with television channels. But I assume you are refering to the identity for each elementary stream.

As for these three use cases, I am not convinced that the 2nd case and 3rd case are two valid cases.
In multiview video case, follow the rules specifed in the section 5.2 of RFC3550, each view
SHOULD be carried in a separate RTP session/RTP stream. This allow the receiver use SSRC to identify each view
and use CNAME to group all the view belong to the same source together.
In according to RFC3550, put the all the views in the same RTP stream is a good design. 

In MPEG Transport Streams, I think we should more focus RTP stream or how MPEG TS is carried over RTP, how RTP reciever 
demultiplex the RTP streams using Session multiplexing specified
in RFC3550 and SSRC multiplexing specified in RFC4588. I am not aware that RTP define any other mechanim to demultiplex each elementary 
stream in one MPEG Transport stream. I am not sure this is XRBLock or AVTCore scope to specify PID multiplexing mechanism or use 
such mechanism specified somewhere else to differenitate each elementary stream in the MPEG Transport Stream?
Also I am not sure we can identify which PID beong to which source unless we have out band signaling specified to indicate such mapping.

So It will be more sensible to limit channel concept only to audio application if we do have ways to identify each channel in the audio application and map each channel to RTP stream.

Regards!
-Qin
  ----- Original Message ----- 
  From: Alan Clark 
  To: [email-masked] 
  Sent: Thursday, November 17, 2011 10:46 AM
  Subject: [xrblock] Multiple channels in RTP streams

  On the question of multiple channels in RTP streams - it may be helpful to have some practical examples to use for reference and as examples.  This may also answer one question in the XRBLOCK session which related to the numbering of the channels in the case of multichannel audio.

  (i) Multichannel Audio
  RFC3551 defines an RTP profile for Audio and Video Conferences.  Section 4 describes the convention for multichannel sample based audio encodings - which is to interleave samples using the orderings

  Channels        1    2    3    4    5    6
  2                      L    R
  3                      L    R    C
  4                      L    C    R    S
  5                      FL  FR  FC  SL  SR
  6                      L    LC   C   R    RC    S

  This is generally consistent with the AIFF-C format, which is widely used for multichannel audio - and uses this interleaved approach.  Note that this does define an ordering, and hence channels 1, 2, 3, 4.. can be interpreted in terms of Left, RIght etc.  Other IETF RFC's that relate to multichannel audio include RFC3190, RFC4184, RFC5691 .....

  (ii) MPEG Transport Streams
  An MPEG Transport stream comprises multiple channels of video and/or audio carried in MPEG Transport packets, with each media stream identified by PID.  A typical single program stream might have, for example, one video stream and two audio streams (for two languages) plus additional streams for captions, contents (PAT/PMT).  This MPEG Transport Stream is carried in a single RTP session.  A typical use case would require the reporting of a Video MOS for the video stream and an Audio MOS for each of the audio streams.
   
  (iii) Multi-view Video
  Multi-view video generally covers 3D, multiple viewpoint video and other similar applications.  Work on this has been performed within MPEG/VCEG as extensions to H.264 and an RTP payload format was proposed in draft-ietf-avt-rtp-mvc. 

  Best Regards

  Alan

------------------------------------------------------------------------------

  _______________________________________________
  xrblock mailing list
  [email-masked]
  https://www.ietf.org/mailman/listinfo/xrblock

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Tue, 22 Nov 2011 14:56:51 +0100

Hi Qin,

This is exactly the point that bothered me in the generalization and
lead to asking the question at the meeting. We use the term 'channel' in
the three different use cases for three things that have commonality but
are not identical, and then we have difficulties in identifying them
uniquely and mapping them to RTP streams. 

Dan

From: [email-masked] [mailto:[email-masked]] On
Behalf Of Qin Wu
Sent: Tuesday, November 22, 2011 3:15 PM
To: Alan Clark; [email-masked]
Subject: Re: [xrblock] Multiple channels in RTP streams

It looks multichannel in 3 cases has different meaning and not sure we
can interpret or abstract it into the channel concept.

E.g., In multiple channel audio case(i.e., stereo audio application),
multi-channel can be categorize into left audio channel, right audio
channel, etc.

In Multiview video case, multiview can be catorized into left view,
right view, etc.

I am not sure we can unify multiview video and multi-channel audio into
channel concept.

In MPEG Transport Stream case, a transport stream used in digital
television might contain multiple programs, to represent multiple
television channels

each channel may consists of multiple video streams, multiple audio
streams, and any necessary metadata, if we take 13 bits PID as channel
ID or identity for each elementary stream(e.g.,

video stream, audio stream), channel ID is quite confusing with
television channels. But I assume you are refering to the identity for
each elementary stream.

As for these three use cases, I am not convinced that the 2nd case and
3rd case are two valid cases.

In multiview video case, follow the rules specifed in the section 5.2 of
RFC3550, each view

SHOULD be carried in a separate RTP session/RTP stream. This allow the
receiver use SSRC to identify each view

In according to RFC3550, put the all the views in the same RTP stream is
a good design. 

In MPEG Transport Streams, I think we should more focus RTP stream or
how MPEG TS is carried over RTP, how RTP reciever 

in RFC3550 and SSRC multiplexing specified in RFC4588. I am not aware
that RTP define any other mechanim to demultiplex each elementary 

stream in one MPEG Transport stream. I am not sure this is XRBLock or
AVTCore scope to specify PID multiplexing mechanism or use 

such mechanism specified somewhere else to differenitate each elementary
stream in the MPEG Transport Stream?

Also I am not sure we can identify which PID beong to which source
unless we have out band signaling specified to indicate such mapping.

So It will be more sensible to limit channel concept only to audio
application if we do have ways to identify each channel in the audio
application and map each channel to RTP stream.

Regards!

-Qin

	----- Original Message ----- 

	From: Alan Clark <mailto:[email-masked]>  

	To: [email-masked] 

	
	On the question of multiple channels in RTP streams - it may be
helpful to have some practical examples to use for reference and as
examples.  This may also answer one question in the XRBLOCK session
which related to the numbering of the channels in the case of
multichannel audio.
	
	(i) Multichannel Audio
	RFC3551 defines an RTP profile for Audio and Video Conferences.
Section 4 describes the convention for multichannel sample based audio
encodings - which is to interleave samples using the orderings
	
	Channels        1    2    3    4    5    6
	2                      L    R
	3                      L    R    C
	4                      L    C    R    S
	5                      FL  FR  FC  SL  SR
	6                      L    LC   C   R    RC    S
	
	This is generally consistent with the AIFF-C format, which is
widely used for multichannel audio - and uses this interleaved approach.
Note that this does define an ordering, and hence channels 1, 2, 3, 4..
can be interpreted in terms of Left, RIght etc.  Other IETF RFC's that
relate to multichannel audio include RFC3190, RFC4184, RFC5691 .....
	
	(ii) MPEG Transport Streams
	An MPEG Transport stream comprises multiple channels of video
and/or audio carried in MPEG Transport packets, with each media stream
identified by PID.  A typical single program stream might have, for
example, one video stream and two audio streams (for two languages) plus
additional streams for captions, contents (PAT/PMT).  This MPEG
Transport Stream is carried in a single RTP session.  A typical use case
would require the reporting of a Video MOS for the video stream and an
Audio MOS for each of the audio streams.
	 
	(iii) Multi-view Video
	Multi-view video generally covers 3D, multiple viewpoint video
and other similar applications.  Work on this has been performed within
MPEG/VCEG as extensions to H.264 and an RTP payload format was proposed
in draft-ietf-avt-rtp-mvc. 
	
	Best Regards
	
	Alan

________________________________

---

From: Qin Wu <[email-masked]>
Date: Wed, 23 Nov 2011 08:48:26 +0800

Multiple channels in RTP streamsOne correction below.
s / "put the all the views in the same RTP stream is a good design. "/ "put the all the views in the same RTP stream is not a good design. "

Regards!
-Qin
  ----- Original Message ----- 
  From: Qin Wu 
  To: Alan Clark ; [email-masked] 
  Sent: Tuesday, November 22, 2011 9:14 PM
  Subject: Re: [xrblock] Multiple channels in RTP streams

  It looks multichannel in 3 cases has different meaning and not sure we can interpret or abstract it into the channel concept.
  E.g., In multiple channel audio case(i.e., stereo audio application), multi-channel can be categorize into left audio channel, right audio channel, etc.
  In Multiview video case, multiview can be catorized into left view, right view, etc.
  I am not sure we can unify multiview video and multi-channel audio into channel concept.

  Regards!
  -Qin
    ----- Original Message ----- 
    From: Alan Clark 
    To: [email-masked] 
    Sent: Thursday, November 17, 2011 10:46 AM
    Subject: [xrblock] Multiple channels in RTP streams

    (i) Multichannel Audio

    Channels        1    2    3    4    5    6
    2                      L    R
    3                      L    R    C
    4                      L    C    R    S
    5                      FL  FR  FC  SL  SR
    6                      L    LC   C   R    RC    S

     
    (iii) Multi-view Video

    Best Regards

    Alan

----------------------------------------------------------------------------

---

From: Qin Wu <[email-masked]>
Date: Wed, 23 Nov 2011 17:13:05 +0800

Multiple channels in RTP streamsYes, 
Suppose using MPEG Transport Stream to carry multi-channel  audios and MPEG TS is carried over RTP, how do we rely on Packet ID (PID) 
in the MPEG TS layer to identify left channel audio and right channel audio and center channel audio? 

I am aware that MPEG TS can use stream_type to differentiate video stream from audio stream . However it is not clear 
to me how one audio stream can be further differentiated into left channel audio stream and right channel audio 
stream? I believe only stereo decoder knows how to distinguish them based on specific codec and order of each channels.
Also SDP signaling will be used to indcate to stereo decoder how many channel are really used.

If multi channel audios can be carried in separate RTP stream, such issue mentioned above can be avoided. Since each channel audio
will use different SSRC.

So the question is do we need to cover the case where multiple channel audios are carried in the same RTP stream? 

Regards!
-Qin
----- Original Message ----- 
  From: Romascanu, Dan (Dan) 
  To: Qin Wu ; Alan Clark ; [email-masked] 
  Sent: Tuesday, November 22, 2011 9:56 PM
  Subject: RE: [xrblock] Multiple channels in RTP streams

  Hi Qin,

  This is exactly the point that bothered me in the generalization and lead to asking the question at the meeting. We use the term 'channel' in the three different use cases for three things that have commonality but are not identical, and then we have difficulties in identifying them uniquely and mapping them to RTP streams. 

  Dan

  From: [email-masked] [mailto:[email-masked]] On Behalf Of Qin Wu
  Sent: Tuesday, November 22, 2011 3:15 PM
  To: Alan Clark; [email-masked]
  Subject: Re: [xrblock] Multiple channels in RTP streams

  Regards!

  -Qin

    ----- Original Message ----- 

    From: Alan Clark 

    To: [email-masked] 

    (i) Multichannel Audio

    Channels        1    2    3    4    5    6
    2                      L    R
    3                      L    R    C
    4                      L    C    R    S
    5                      FL  FR  FC  SL  SR
    6                      L    LC   C   R    RC    S

     
    (iii) Multi-view Video

    Best Regards

    Alan