Subject: [AVT] ATRAC Family Payload update

From: Matthew Romaine <[email-masked]>
Date: Wed, 18 Feb 2004 13:22:50 +0900

Magnus and the avt crew,

I have compiled comments thus far regarding the new ATRAC payload 
format, and have attempted to resolve each issue.  Please let me know 
if the responses are heading in the right direction.

Magnus wrote:  "There is need to resolve how the setup signalling for 
this payload format will work. The normal model is that one defines a 
MIME type..."

A few new sections have been added to the draft, which I have inserted 
below.  Is this along the lines of what is needed?

5.1  ATRAC Family MIME Registraion

   The MIME subtype for the Adaptive TRansform Codec Family (ATRAC
   Family) is allocated from the Vendor tree since ATRAC is intended to
   be used with commercial products, and use of ATRAC requires a license
   from Sony Corporation, the vendor.

   Note, any unspecified parameter MUST be ignored by the receiver.

   Media Type name:  audio
   Media subtype name:  ATRACFAMILY
   Required parameters:
     profile:             ATRAC, ATRAC3, ATRAC-X
     frameSampleSize:     512, 1024, 2048
     sampleRate:          32000, 44100, 48000, 88100, 96000
     frameLength:         [license required]
     channels:            1, 2, 3, 4, 5, 6, 7, 8

   Optional parameters:
     maxptime: The maximum amount of media which can be encapsulated in
       a payload packet, expressed as time in milliseconds.  The time is
       calculated as the sum of the time the media present in the packet
       represents.  The time SHOULD be a multiple of the frame size.  If
       this parameter is not present, the sender MAY encapsulate a
       maximum of 16 encoded frames into one RTP packet.

     ptime:    see RFC2327 [2]

   Encoding considerations:
     This type is defined for transfer via RTP (RFC 1889).

   File extensions: at1, atr, at3, atx
   Macintosh file type code: none
   Object identifier or OID: none

   Intended usage: LIMITED USE
     Only licensees of ATRAC technology may use this type.

5.2  Mapping MIME Parameters into SDP

   The information carried in the MIME media type specification has a
   specific mapping to fields in the Session Description Protocol (SDP)
   [2], which is commonly used to describe RTP sessions.  When SDP is
   used to specify sessions employing the ATRAC family of codecs, the
   mapping is as follows:

   -  The MIME type ("audio") goes in SDP "m=" as the media name

   -  The MIME subtype (payload format name) goes in SDP "a=rtpmap" as
      the encoding name.  The RTP clock rate in "a=rtpmap" MUST be as
      indicated in the required parameter "sampleRate".

   -  The parameters "ptime" and "maxptime" go in the SDP "a=ptime" and
      "a=maxptime" attributes, respectively.

   -  Any remaining parameters go in the SDP "a=fmtp" attribute by
      copying them directly from the MIME media type string as a
      semicolon separated list of parameter=value pairs.

   Some example SDP session descriptions utilizing ATRAC encodings
   follow.  In these examples, long a=fmtp lines are folded to meet the
   column width constraints of this document; the backslash ("\") at the
   end of a line and the carriage return that follows it should be
   ignored.

   Example usage of ATRAC with stereo at 44100Hz encoded with ATRAC3:

     m = audio 49120 RTP/AVP 99
     a = rtpmap:99 ATRACFAMILY/44100
     a = fmtp:99 profile=ATRAC3; frameSampleSize=1024; frameLength=312; \
         channels=2
     a = maxptime:20

   Example usage of ATRAC with 5.1 setup at 48000Hz encoded with ATRAC-X:

     m = audio 49120 RTP/AVP 99
     a = rtpmap:99 ATRACFAMILY/48000
     a = fmtp:99 profile=ATRAC-X; frameSampleSize=2048; \
         frameLength=1156; channels=6
     a = maxptime:50

(An Offer/Answer Model consideration is still being drafted.)

Magnus wrote: "I would like to see some more information about the 
codecs in the beginning of the draft. Especially properties like, 
possible sampling frequency, multi-channel support, frame sizes, etc. 
which effects the payload design."

Here is how the details section in the introduction now reads:

   Early versions of the ATRAC codec handled only two channels of audio
   at 44.1kHz sampling frequency, with typical bit-rates between 66kbps
   and 132kbps.  The latest version allows for up to 8 channels of audio
   at 96kHz sampling frequency.  The feasible bit-rate range has also
   expanded to from 8kbps to 1400kbps.

   Depending on the version of ATRAC used, the sample-frame size is
   either 512, 1024 or 2048.  Actual bit-rates are determined by
   specifying a fixed encoded frame-size.  In other words, instead of
   requesting a stereo 44.1kHz stream at, say, 64kbps, one would tell
   the encoder to create encoded frame-sizes of 364bytes.

Magnus wrote:  "The Frame Offset fields is not sufficiently defined. I 
don't understand how you can determine the TS offset for each frame in 
the packet using only this field. Also how does it work when there are 
more than 2 frames present in the same RTP payload?"

Here is how the new Frame Offset section reads:

   Frame Offset (FrOff): 4 bits
   The purpose of frame offsets is to provide a basic mechanism for the
   transmission of redundant data.  Using the current packet's timestamp
   as a reference, the frame offset is converted to units of the
   timestamp.  Since frame-size lengths are fixed after being determined
   during SDP initiations (one of either 512, 1024, or 2048 samples),
   the playback time of the first frame (in relation to the current
   timestamp) in the packet can then be calculated.

   As an example of using Frame Offsets, consider a situation when FrOff
   is 3.  If a packet has 5 frames of audio, with each frame
   representing 1024 samples of audio, then we calculate that playback
   begins 3 frames (3072 samples) before the current timestamp.  (The
   only other necessary variable is sampling frequency, which MUST have
   been established during SDP negotiations).  This field SHOULD NOT be
   used in packets containing fragmented data.

Magnus wrote: "The security section will need more work. I think there 
is need to discuss also authentication and integrity protection. Also 
congestion control might need some special consideration, due to 
redundancy transmission and possibility to vary bit-rate."

Is a model similar to what exists in the AMR-WB+ draft sufficient?

thanks,
matt

----------------------------------------
Matthew M.I. Romaine
IMNC Audio Codec Group
Sony Corporation
phone: 03-5448-6065
fax: 03-5448-5617
----------------------------------------

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 23 Feb 2004 14:12:30 +0100

Hi Matthew,

Here is some comments on the proposed update.

> below.  Is this along the lines of what is needed?

Yes, you definitely need a MIME registration.  More comments about 
details inline. In general I thing all the required parameters needs a 
bit more text.

>      channels:            1, 2, 3, 4, 5, 6, 7, 8

The question is if it would be a better alternative to define three MIME 
types, one for each of the profiles. I assume that you have plans to 
allow for new profiles, thus one needs to consider backwards 
compatibility etc. In this case it is probably easier to define one MIME 
type for each profile. Thus a receiver will not need to look into FMTP 
line to know if it supports the codec profile or not. One will also 
avoid having backwards compatibility issues for any new codec if it 
needs new parameters.

Having a parameter that requires license does not look good in my eyes. 
It needs to at least define what possible values that it can have. If 
the exact values are secret defines what range that is allowed.

Also does a specific channel assignment rule for the number exist? Are 
you using the one in RFC 3551 with extension for 7 and 8 channels?

Also I think you should consider how you intend to use the frame sample 
  size and the frame length parameters. In what directions does they in 
fact apply between a sender and receiver?

>      This type is defined for transfer via RTP (RFC 1889).

The RTP reference to use is RFC 3550.

>    File extensions: at1, atr, at3, atx

Are you registering a file format at the same time. The file extension 
is in relation to the a file format registration. I think you will need 
to clarify how the file format part relates to the RTP payload.

>       indicated in the required parameter "sampleRate".

No, I would specify this the other way. The MIME parameter sampleRate 
goes as the RTP clock rate in the "a=rtpmap" attribute. Also the 
channels should probably go in "rtpmap" attribute as an the first 
encoding parameter.

How common is it expected to change the number of channels in a RTP 
stream, i.e. for the same SSRC? If it possible to change at all, have 
you considered the problems, see my mail about this.
http://www1.ietf.org/mail-archive/working-groups/avt/current/msg03384.html

> (An Offer/Answer Model consideration is still being drafted.)

Yes, this one will be very important, especially as you have a number of 
required parameters.

>    the encoder to create encoded frame-sizes of 364bytes.

Thanks, that help somewhat.

>    used in packets containing fragmented data.

The problem I see is that Frame Offset if used needs to be defined for 
each frame in the payload. However what I understand it only occurs one 
in the payload header. Thus I can't understand why one can't use the 
timestamp to place the data correct.

I would also think that some text on how one aggregates multiple frames 
and derive their timestamps are needed.

When sending multiple channels are they in a single block of data or are 
they sent in multiple blocks of data, each contains a channel or two? If 
  they are split, how does one determine which channel the data belongs to.

> Is a model similar to what exists in the AMR-WB+ draft sufficient?

Yes, I would say so. However there are need to consider if the codec 
itself contains any security problems, like possibility to create 
bit-streams that are much more resource demanding than common data. I 
would also recommend that you looks at RFC 3552.
ftp://ftp.rfc-editor.org/in-notes/rfc3552.txt

Cheers

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

This communication is confidential and intended solely for the addressee(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you believe this message has been sent to you in error, please notify the sender by replying to this transmission and delete the message without disclosing it. Thank you.

E-mail including attachments is susceptible to data corruption, interruption, unauthorized amendment, tampering and viruses, and we only send and receive e-mails on the basis that we are not liable for any such corruption, interception, amendment, tampering or viruses or any consequences thereof.

---

From: Matthew Romaine <[email-masked]>
Date: Tue, 24 Feb 2004 19:12:16 +0900

Magnus,

> you using the one in RFC 3551 with extension for 7 and 8 channels?

The channel assignments charted in RFC 3551 don't really work for us,  
since for one thing an LFE channel is not specified.  So I'll add a  
similar chart for channel assignments in the draft.  Thanks for  
pointing this out.

> Also I think you should consider how you intend to use the frame  
> sample  size and the frame length parameters. In what directions does  
> they in fact apply between a sender and receiver?

This information is needed to initialize the decoder on the receiving  
side; rather than send it every packet, we thought making it a MIME  
parameter would allow for its use during SDP negotiations.  Is there  
something more we need to consider?

> http://www1.ietf.org/mail-archive/working-groups/avt/current/ 

Our current thinking is to use translators and perhaps mixers to handle  
changes in the number of channels.  Our initial payload proposals which  
we have since dropped did try to allow for channel changes mid-stream,  
but we have decided to forgo that feature.  Basically, we were trying  
to project a payload format that assumed a scalable codec onto a  
non-scalable one (in the strictest sense, anyways).  So at the moment  
we do not plan on having the number of channels change w/o a  
re-negotiation via SDP.

> each frame in the payload. However what I understand it only occurs  
> one in the payload header. Thus I can't understand why one can't use  
> the timestamp to place the data correct.

Just to be clear, multiple audio frames in a packet (with a maximum of  
16) must be stored sequentially.  So I'm not quite sure why you think  
each frame needs the FrOff field....hm... Since our timestamp will be  
sample based, our initial thinking was to have sequential packets  
increase their timestamp by the number of samples sent in the previous  
packet.  We were thinking the timestamp should basically not be  
manipulated beyond increasing it as necessary for each succeeding  
packet (shouldn't there be at least one field that is atomically  
increasing?)  Thus, when sending redundant frames, we have a field to  
indicate how many 'previous frames' are being sent instead of changing  
the timestamp.  Since a packet can only hold up to 16 audio frames  
(where a single frame can hold up to 1 to 8 channels of encoded audio  
-- see next response), there is a maximum difference possible between  
the "current timestamp" and the largest allowable "rewind".  By doing  
this, we ensure the stream will progress; if we allow manipulations of  
the timestamp, who knows what some crazy implementor may do :)  Also  
keep in mind that we are allowing previous frames to be followed by new  
frames, all in the same packet.  Maybe this ASCII visualization will  
help:

|---Fr1--|--Fr2--|--Fr3--|--Fr4--|--Fr5--|       Nth Packet

                           |--Fr3--|--Fr4--|--Fr5--|--Fr6--|--Fr7--|     
N+1th Packet

In this case, the "N+1th" packet would have it's FrOff field = 3, since  
the timestamp in packet "N+1" would indicate where Fr6 should start.

I suppose there are a few ways to do the same thing.... is there  
something we are missing with this approach?

> When sending multiple channels are they in a single block of data or  
> are they sent in multiple blocks of data, each contains a channel or  
> two? If  they are split, how does one determine which channel the data  
> belongs to.

multiple channels in any variety are sent as single blocks of data;  
there is no interleaving of encoded audio frames, so there is no need  
to worry about which "block of data" is "which channel".  This will  
probably be a point of confusion for anyone who keeps our previous (now  
dropped) payload format in mind.  We have completely scrapped that  
implementation.  Our main goal at this point is to keep things very  
simple while still allowing streaming of multichannel data.  I will  
clarify this in the draft (as well as how timestamps are determined,  
etc.)

btw, when would you like to schedule our meeting outside the avt  
session?
Oh, and by when should we get you our presentation material?

Thanks again,
Matt

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 25 Feb 2004 16:40:01 +0100

Hi,

comments inline.

> similar chart for channel assignments in the draft.  Thanks for  

Good,

> something more we need to consider?

Not knowing enough about the codec I can't determine what in my opinion 
would be the most suitable place for this information. If the frame 
sample size normal is fixed for a transaction then having it encoded 
through the payload is the correct choice. Otherwise some other 
indication might be more appropriate.

> we do not plan on having the number of channels change w/o a  

As you do not intend to have channel changes being common I think the 
one PT per channel configuration is the correct choice. Thus one can 
define multiple payload types that maps to different channel 
configurations. Thus a receiver will know of the possibility of 
switching channel information mid session. Of course if one declares 
only one and later needs to change one can in some of the application 
update the SDP.

> something we are missing with this approach?

Okay, now I understand your approach. I think the main problem with this 
approach is that the RTP timestamp does not reflect the oldest data in 
the packet. I think there are more simple ways informing the client what 
buffer needs a stream with redundancy would need. So I would be more 
interested in using a TS that actually points on earliest frame in the 
packet. If you are looking on your example from a TS perspective it 
looks like this:

#1  #2  #3  #4  #5                   TS = 4
         #3  #4  #5  #6  #7           TS = 6
                 #5  #6  #7  #8  #9   TS = 8

If one uses a earliest possible scheme it would instead look like:

#1  #2  #3  #4  #5                   TS = 1
         #3  #4  #5  #6  #7           TS = 3
                 #5  #6  #7  #8  #9   TS = 5

In this case one could signal max-overlapp=3 to tell the receiver that 
this may contain redundant data up to 3 frames old, and thus buffer for 
a minimum of 3 frames to be able to take advantage of the data.

So in my opinion this bits are basically wasted. They do not give any 
advantage which can't not be achieved through other means without 
spending bits in the payload header.

Cheers

Magnus Westerlund