Subject: MPEG-4 (including Flexmux), over RTP/RTSP

From: Dave Singer <[email-masked]>
Date: Fri, 19 May 2000 11:13:21 -0700

This is a portmanteau message, trying to cover the whole space of 
carriage of MPEG-4 in IETF protocols.  I'm trying to pull everything 
together.

It started by the resurgence of comment on FlexMux in RTP, so I'll 
start with packing formats.

PACKING FORMATS

I have been thinking about packing formats for MPEG-4 data, and I 
feel that we need to be a little more flexible.  I'd suggest that at 
least the following have value:

a) a simple packing of a single, arbitrary, elementary stream into 
RTP packets.  The packing would not be stream-format-aware, and so 
could carry any one ES.  [MPEG4-GENERIC]

b) a packing of MPEG-4 video, with care to place RTP packet 
boundaries at sub-frame boundaries, such that partial frame display 
is possible at the reciever.  I suspect that the video group's "video 
packet" meets the need, and may not require any RTP-specific header. 
However, it should be a distinct payload name so that receivers can 
know that video-aware packetization has happened.  [MPEG4-VIDEO]

c) a packing of MPEG-4 audio which interleaves the audio frames.  See 
the existing specifications for QCELP and the draft from Ross 
Finlayson for MP3 for examples.  Many audio decoders can do some 
sensible fill-in for single missing frames, but this fill-in error 
concealment rapdily gets worse if multiple consecitive frames are 
lost.  By intrerleaving, the decoder gets a sporting chance to 
conceal errors.  A small RTP header is needed to indicate that this 
has happened.  (With a suitable choice of header, it's possible that 
packing (a) could be a special case of this.)  [MPEG4-AUDIO]

d) a packing of flexmux into RTP.  This enables the carriage of large 
numbers of streams, where the use of separate RTP streams becomes 
prohibitive.  [MPEG4-FLEXMUX]

It may be that some formats become common enough, and have such 
characteristics, that a unique payload format for it becomes 
warranted (e.g. AAC).  That should be possible down the road.

For those streams requiring reliable delivery, I'd like to see us 
leverage existing work in the IETF in this area (including, but not 
limited to FEC, re-transmission, or repetition), rather than making 
it a characteristic of the packing scheme itself.

I'd like to think that all these, though separate payload schemes 
with separate names (in the RTPMAP) can be otherwise treated 
uniformly, in terms of timescale and timestamp, ES-ID to RTP mapping, 
and so on.

TIME-STAMP HANDLING

I think we are all agreed on this: that the MPEG-4 timescale for an 
ES becomes the RTP timescale, that the composition time maps to the 
RTP time, and the decode time maps to transmission order.

SDP INFORMATION

I think we can assume, for the moment, that any session described by 
SDP (e.g. in SAP, as a file download, as a DESCRIBE over RTSP) has at 
most one MPEG-4 session, though I'd like to see how we could lift 
this restriction.

We need
i) RTP stream to elementary stream mapping.  This needs to cover the 
Flexmux case as well as the single stream.
a=x-mpeg4-esid a.b.c
or
a=x-mpeg4-esids m.i:a.b.c,n.k:p.q

The first attribute is used for single streams; the second for 
flexmux.  In this, a is the ESID of the top-level OD stream, b is the 
ESID within that scope of another OD stream, and c is the ESID within 
that scope of the stream;  similarly for p and q.  m and n are 
muxcodes (identifying the table used), and i and k are flexmux 
channel numbers within the indicated muxcode.

The entire muxcode table(s) also need to be transferred.  For the 
moment, I propose another attribute for them.  Since these tables 
have a length indicator, multiple tables can be concatenated 
together, if required.
a=x-mpeg4-muxcodetable <location>

where <location> is a URL that will supply the table(s).  If they are 
small, a DATA: URL will probably suffice to carry them in-line. If 
not, the URL should use a file-retrieval scheme (e.g. HTTP, FTP). 
The data at the indicated URL consists of some number of muxcode 
tables, complete, in binary format (but note that DATA URLs allow for 
base64 encoding of binary data, which would be needed here).  The 
mime type of a muxcode table needs deciding 
(application/x-mpeg4-muxcodetable?).

ii)  Some indication that an MPEG-4 session is carried, and the means 
to get the IOD, if the terminal doesn't already have it by some other 
means.
In SDP, as a session-level attribute:
      a=x-mpeg4-iod [<location>]
      In an RTSP session, the location is optional.  If not supplied, 
the IOD is retrieved over the RTSP session by using either DESCRIBE 
with an accept of type application/x-mpeg4-iod, or a GET-PARAMETER 
(we need to decide which). Where the SDP information is supplied by 
some other means (e.g. as a file, in SAP), the location is obligatory 
and should be a URL which will supply the IOD (e.g. small ones may be 
encoded using DATA:, or otherwise HTTP: or other suitable file-access 
URL).
-- 
David Singer
Apple Computer/QuickTime