Subject: [AVT] AVT opinions required on Vorbis over RTP

From: Tor-Einar Jarnbjo <[email-masked]>
Date: Fri, 02 Sep 2005 19:54:59 +0200

Hi,

the discussions around the proposed RFC for Vorbis over RTP are going 
again on the vorbis-rtp mailing lists, and I was just wondering if 
someone from avt could comment on one the issues being currently discussed.

One of the main problems with Vorbis is that each stream contains a 
codebook header, which has to be reliably transmitted to the client, to 
allow the client to decode the rest of the actual audio stream. Some of 
the guys on the vorbis-rtp mailing list are now proposing in-band RTP 
delivery without client acknowledge at the beginning of the RTP session 
as the only mandatory codebook delivery mechanism to be required by the 
RFC with additional reliable delivery mechanisms only being an optional 
extension.

To me, this solution seems pretty useless and I was now hoping that 
someone from AVT could take time to give a short estimate, if an RFC 
proposal requiring reliable delivery of some data over RTP will have any 
chance at all to be approved. I would assume not.

Regards,

Tor-Einar Jarnbjo

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: "Dan Wing" <[email-masked]>
Date: Fri, 2 Sep 2005 11:19:49 -0700

If the codebook is only sent at the beginning of the call, could you maybe
create "Codebook Preconditions", patterned after connectivity preconditions
(draft-ietf-mmusic-connectivity-precon), to ensure the codebook is delivered
before the remote endpoint is alerted and the call completes?

-d

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Tor-Einar Jarnbjo
> Sent: Friday, September 02, 2005 10:55 AM
> To: [email-masked]
> Subject: [AVT] AVT opinions required on Vorbis over RTP
> 
> Hi,
>

---

From: "Stephan Wenger" <[email-masked]>
Date: Fri, 2 Sep 2005 21:34:53 +0300

Hi Tor-Einar, folks,

I won't comment on the chances of approval of anything in AVT, but let me
state that we had to solve a very similar problem in RFC3984, and did it as
follows:

In H.264, we have data structures known as parameter sets, which are very
sensitive to errors and which need to be available before the decoding of
the actual slice data starts.  I guess that makes them comparable to a
vorbis codebook.  Likely they are smaller; perhaps much smaller, though.  I
don't know vorbis enough to make a judgment here.  A typical parameter set
is only a couple of bytes, but it can grow to several hundred bytes per set,
and we need more than one (perhaps many) of them.

Those parameter sets can be conveyed in-band or out of band.  When they are
conveyed in-band, each parameter set goes into its own packet, or a number
of them are aggregated in one packet.  The RFC suggests sending redundant
copies so to increase the chance of reception.  

We utilize SDP for out-of-band sending, please search RFC 3984 for
sprop-parameter_set for a discussion how to do this.  As we use BASE64
encoding, the overhead is reasonable -- 33% over the binary.  

System standards utilize both mechanisms.  H.241, for example, relies of
in-band transmission (H.241 is used in conjunction with H.324(M) and H.323
video conferencing).  Some 3GPP packet switched services, on the other hand,
employ the out-of-band mechanism.

So perhaps this model is an option for vorbis.  It's certainly in line with
common standardization principles -- when in doubt, do it both.

(The latter comment may be not politically correct in the IETF, but it still
holds in practice :-)

Stephan

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Tor-Einar Jarnbjo
Sent: Freitag, 2. September 2005 20:55
To: [email-masked]
Subject: [AVT] AVT opinions required on Vorbis over RTP

Hi,

Regards,

Tor-Einar Jarnbjo

---

From: Tor-Einar Jarnbjo <[email-masked]>
Date: Fri, 02 Sep 2005 20:50:29 +0200

Hi Stephan,

>and we need more than one (perhaps many) of them.
Perhaps I should have explained the details a bit more thrououghly. The 
Vorbis codebook size depends on the encoder and have (in theory) no size 
limitation. The reference encoder creates codebooks at around 5kB for 
common bit rates. I think SDPs are recommended to be kept below 1kB, so 
including the codebook header in the SDP is not an option and I am 
concerned that repeating the codebooks at the session beginning will 
result in unacceptable delays before audio transmission can start. We 
must also try to find a solution which will work in multicast situations.

Regards, Tor

---

From: Ross Finlayson <[email-masked]>
Date: Fri, 02 Sep 2005 12:09:47 -0700

Let's face it - Vorbis's use of dynamic codebooks is a major 
Archilles heel for streaming.  One could even go so far as to call it 
an architectural flaw.  (Having to send a complete new codebook 
potentially for each song partially defeats the purpose of audio compression.)

It would be much better to have a fixed, predefined set of codebooks 
that all receivers would know in advance.  Then you'd need to send 
only a 'codebook id', which should be small enough to include in the 
Vorbis RTP payload format header.    E.g., if there were <= 256 
pre-defined 'codebooks for streaming', then you could use a 1-byte 
field within the RTP payload format to indicate which one.

I suggest that the Vorbis designers should just figure out a complete 
set of codebooks that are likely to be useful, and use only those for 
streaming.

Some other people's comments on this issue (previously on the "avt" 
mailing list):

At 05:14 PM 2/26/01, Dave Singer wrote:
>This being the case, I suggest the approach that JPEG took:  make it 
>possible to transmit the code-bbooks in stream, but also have a set 
>that are statically defined and that can be selected simply by 
>index. Despite what you say about them changing with every release 
>below, I bet that by the time the RTP payload is mature, there can 
>also be a mature set of codebooks...

At 01:12 PM 7/9/03, Aaron Colwell wrote:
>I agree with Ross on this. It seems that these codebooks are causing too
>many problems that aren't worth dealing with. People have been suggesting
>getting the codebooks via HTTP, alternate TCP links, and all sorts of
>other methods. If your going to go that far why not just use HTTP to
>deliver the data. That way you are guaranteed to get everything in the
>right order.
>
>My guess is that other video and audio codecs use predefined tables to
>avoid this very problem. Vorbis should do the same. The Vorbis folks
>should come up with a "streamable" profile that encodes the audio using
>predefined codebooks. Doing this makes all of the complicated codebook
>issues go away. They could also create an offline tool that will translate
>any Vorbis file into a "streamable" Vorbis file. This idea is similar to
>hinting tools used for MPEG4 and 3GPP files.

---

From: Colin Perkins <[email-masked]>
Date: Sat, 3 Sep 2005 01:31:17 +0100

On 2 Sep 2005, at 19:50, Tor-Einar Jarnbjo wrote:
>> is only a couple of bytes, but it can grow to several hundred  
>> bytes per set,and we need more than one (perhaps many) of them.
>>
>>
> Perhaps I should have explained the details a bit more thrououghly.  
> The Vorbis codebook size depends on the encoder and have (in  
> theory) no size limitation. The reference encoder creates codebooks  
> at around 5kB for common bit rates. I think SDPs are recommended to  
> be kept below 1kB, so including the codebook header in the SDP is  
> not an option and I am concerned that repeating the codebooks at  
> the session beginning will result in unacceptable delays before  
> audio transmission can start. We must also try to find a solution  
> which will work in multicast situations.

The 1kB recommendation is for SDP carried in SAP, since it has to fit  
within a single UDP packet. If the SDP is delivered by other means,  
e.g. SIP, RTSP, web page, then there is no size limit (although,  
clearly, smaller is better if you want to reduce setup delays).

Colin

---

From: Marshall Eubanks <[email-masked]>
Date: Mon, 5 Sep 2005 11:53:45 -0400

Dear Ross;

> potentially for each song partially defeats the purpose of audio compression.)

At least the last time I looked into this, you could have to send  a
new cookbook for
each audio _frame_ (i.e., a song could mix cookbooks inside it).

I have always thought and still don't see why RTP Vorbis couldn't be a
subset of
Vorbis with

- a fixed cookbook per stream (like a Mpeg-4 stream) and
- some out of band means of getting the cookbook plus 
-- caching of cookbooks

Then the SDP could just contain (say) a MD-5 hash of the cookbook in
use, with a pointer
on how to get it if it doesn't match one you have,

Regards
Marshall

---

From: Tor-Einar Jarnbjo <[email-masked]>
Date: Mon, 05 Sep 2005 23:24:28 +0200

>each audio _frame_ (i.e., a song could mix cookbooks inside it).
That's not entirely correct - a single Vorbis stream will use the same 
codebook setup for all its audio frames. The problem with different 
codebooks is related to the Ogg feature called chaining. Ogg is the 
container format usually used for Vorbis streams and Ogg defines a 
feature called chaining or call it stream concatenation if you like. You 
can chain as many streams in an Ogg file as you like, each stream 
potentially having completely different properites, be it the codebook 
header, sample rate or number of channels for two different two Vorbis 
streams or for that sake two streams with different content types. Some 
people don't seem to understand the difference between Ogg and Vorbis 
and are now trying to mix container and stream functionality to keep Ogg 
features when transmitting "Ogg-less" Vorbis over RTP. To me it would be 
a logical solution to demand separate RTP sessions for each stream 
within an Ogg container or for that sake only allow streams with 
identical properties to be sent within one RTP session.

>on how to get it if it doesn't match one you have,
This was also more or less the stand of Phil Kerr's last draft, but new 
people have taken over the work on the draft. Luca Barbato published the 
latest version with his modifications yesterday, so I haven't had time 
yet to read it thoroughly:

http://www.j-ogg.de/rfc/vorbis-rtp.html

I'm still keen on hearing any opinions from Xiph or from Fluendo on the 
new direction of the Vorbis/RTP draft.

Tor

---

From: Ross Finlayson <[email-masked]>
Date: Mon, 05 Sep 2005 14:50:16 -0700

>That's not entirely correct - a single Vorbis stream will use the 
>same codebook setup for all its audio frames. The problem with 
>different codebooks is related to the Ogg feature called chaining. 
>Ogg is the container format usually used for Vorbis streams and Ogg 
>defines a feature called chaining or call it stream concatenation if 
>you like. You can chain as many streams in an Ogg file as you like, 
>each stream potentially having completely different properites, be 
>it the codebook header, sample rate or number of channels for two 
>different two Vorbis streams or for that sake two streams with 
>different content types.

Could one of the 'Xiph' people please - once and for all - give us a 
straight answer to the following question:

How many different codebooks are commonly being used for Vorbis audio 
streams?  I don't mean how many codebooks could *potentially* be 
used, but how many codebooks are being used *in practice*, and of 
these, how many are being used frequently.

I imagine that most (if not all?) Vorbis encoders have been 
implemented from a single source code base.  Does this source code 
has a set of pre-defined codebooks for users to choose from?  If so, 
how many such codebooks are there, and how often (if at all) are 
users deciding to use some other codebooks to encode their streams?

I still find it difficult to believe that we cannot define, in 
advance, a small set of codebooks for streaming, and use only those 
codebooks (indicated by a small id) for Vorbis RTP streams.

         Ross.

---

From: Aaron Colwell <[email-masked]>
Date: Tue, 6 Sep 2005 07:42:48 -0700

On Mon, Sep 05, 2005 at 11:24:28PM +0200, Tor-Einar Jarnbjo wrote:
> Some 

Unfortunately the current Vorbis RTP discussions are having a little bit of a
problem with using consistent terms. The current "chaining" discussion really
has more to do with allowing rate adaptation than supporting chaining in the
Ogg sense. Apparently with Vorbis you actually have to change the codebook to
get a lower bitrate. In the simplest cases you could also use the mechanism 
being discussed for some chained files, but it's not a solution that would
work for all valid chained files. For cases where the number of streams changes
, the codecs change, and/or the sample rate changes you'll need to negotiate
multiple RTP sessions.

Aaron