Subject: [AVT] SILK Speech Codec and RTP Payload Format Drafts

From: "Julian Spittka" <[email-masked]>
Date: Tue, 7 Jul 2009 22:10:04 -0700

Dear All,

We have recently submitted two Internet Drafts

draft-vos-silk-00.txt
(https://developer.skype.com/silk?action=AttachFile&do=get&target=draft-vos-
silk-00.txt) 

and 

draft-spittka-silk-payload-format-00.txt
(https://developer.skype.com/silk?action=AttachFile&do=get&target=draft-spit
tka-silk-payload-format-00.txt). 

These were submitted manually since we had problems with the web submission
and may take a few days to show up on the mailing list. We are proposing
SILK as a candidate codec for the Internet Wideband Audio Codec BOF in
Stockholm. We've submitted an initial description of the SILK codec as well
as a payload format draft. Comments are welcome.

Kind Regards,

Julian.

---
Julian Spittka | Skype

Skype me: julian_spittka
email:    [email-masked]

Get Skype for free from www.skype.com
We're hiring http://jobs.skype.com

This electronic message contains information from Skype, which may be
privileged or confidential. The information is intended for use only by the
individual(s) or entity named above. If you are not the intended recipient,
be aware that any disclosure, copying, distribution or use of the contents
of this information is strictly prohibited. If you have received this
electronic message in error, please delete it and notify me, using the
contact details above, immediately.

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 8 Jul 2009 07:59:48 +0200

Julian,

the SILK codec document says:

> SILK can switch in real-time between audio sampling rates of 8, 12, 16
> and 24 kHz.

But the payload spec appears to fix the rate:
-- the SDP just has a single rate parameter
-- using the sample clock directly as the timestamp clock makes it  
hard to switch between rates.

It appears to me you should be using a 48 kHz timestamp clock to  
facilitate
switching.

Gruesse, Carsten

---

From: "Julian Spittka" <[email-masked]>
Date: Wed, 8 Jul 2009 22:51:57 -0700

Hallo Carsten,

Thank you very much for your feedback!

The thought behind only allowing one sample rate value per dynamic payload
type is to use the payload type itself to indicate a sample rate switch in
the RTP header during a call. The SILK decoder will be able to process the
payload encoded at any allowed sample rate without reinitialization or
discontinuity of the audio signal. All allowed modes, i.e. sample rates,
must  be announced separately in the "m=audio" line during call setup.

Regards,

Julian.

-----Original Message-----
From: Carsten Bormann [mailto:[email-masked]] 
Sent: Tuesday, July 07, 2009 11:00 PM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [AVT] SILK Speech Codec and RTP Payload Format Drafts

Julian,

Gruesse, Carsten

---

From: Randell Jesup <[email-masked]>
Date: Thu, 09 Jul 2009 08:54:56 -0400

>must  be announced separately in the "m=audio" line during call setup.

The problem is that there are REAL problems in practice with
timestamp-rate switching in mid-stream.  This is already an unsolved
problem with any stream that switches between (say) G.711 and G.722.
Part (not all) of the problem comes when packets right around the switch
are lost.  If you don't know EXACTLY when the switch happened, your
timebase may be off until the next RTCP packet (and may not be good
until you've had two).  Search the AVT archives for discussion in the
last couple of years of timestamp rate switching; there've been 2 or 3
threads on it.  Also, you may end up triggering actions you don't intend
on a payload change - clients tend to be very active around a payload
change, tearing things down and restarting them, and do all sorts of
non-mandated actions designed to make things work better in practice or
to deal with an unknown source of the change (they might temporarily
mute, reinitialize jitter buffers, drop AV/sync, etc).

This isn't to say it can't be used - but there are a bunch of gotchas
with timestamp rate changes that can be avoided by keeping to a single
rate.

Note that some of the issues mentioned about payload changes may still
apply, so you'll want to think hard about that too.  It's probably
better to signal it in-band in the packets somehow (sometimes it can be
done implicitly using sizes, etc).

>_______________________________________________
>Audio/Video Transport Working Group
>[email-masked]
>https://www.ietf.org/mailman/listinfo/avt

-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS team
[email-masked]
"The fetters imposed on liberty at home have ever been forged out of the weapons
provided for defence against real, pretended, or imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)

---

From: Gregory Maxwell <[email-masked]>
Date: Thu, 9 Jul 2009 07:54:07 -0700

> timestamp-rate switching in mid-stream.  This is already an unsolved

In principle it's possible to hide the sample rate changes inside the codec.

For example, you could signal at the highest rate you would intend to operate. The codec could then be free to downsample internally based on the bitrate targets or other information hten include or imply the current rate with the inband headers and the RX side could upsample as required based on the clients desired output rate. Of course, this works best when the rates in question are related by small common factors.

Speex does something like this already with narrow band vs wideband modes. (A narrow band receiver can get a wideband frame and it will happily output narrow band audio; while a wideband RX will take narrow band input and effectively upsample it to the wideband rate)

Implementations which can easily change their DAC rate on the fly and avoid the computational burden of resampling would be free to do so without any change to the over-the-wire (and thus interoperation relevant) behaviour.

---

From: Randell Jesup <[email-masked]>
Date: Thu, 09 Jul 2009 13:12:42 -0400

Gregory Maxwell <[email-masked]> writes:
>For example, you could signal at the highest rate you would intend to
>operate. The codec could then be free to downsample internally based on
>the bitrate targets or other information hten include or imply the
>current rate with the inband headers and the RX side could upsample as
>required based on the clients desired output rate. Of course, this
>works best when the rates in question are related by small common
>factors. 

You're talking about a slightly different issue (trans-rating, kinda).

The real issue is that RTP timestamps never have had to be directly
equal to the sampling clock, at least not in general.  It's handy
if they are for some uses, but they can also be multiples of it, or be
separate from the actual sample clock (if the payload spec allows).

So, for this usage, you can have a common multiple of the sampling rates
(like 48000) be the timestamp rate, so payload changes don't have to
change the timestamp rate.

The unsolved problem is how to combine in one m=audio(etc) line payloads
with different timestamp rates, like G.711 and G.722.  You could use
PCMU/16000, but just about everyone would choke on that because they
don't expect it.  You can do it (mix rates), but then as mentioned
transitions get funky, and the safest way for a client to deal with it
is to virtually re-initialize everything as if it were a newly connected
stream.  This works, but it has negative effects, and everyone does it
slightly differently (if they do it at all).

Someone proposed a few years back (and was beaten down) using a single
very high timestamp rate for all audio codecs (192K?)

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 09 Jul 2009 13:24:55 -0400

Hi,

What I think Greg was referring to is the idea of making the sampling rate 
switching completely transparent to the signalling. I don't know enough 
details about SILK, so I'll give an example of what we *could* have done 
(but didn't do) if we had wanted to do that with Speex.

Speex supports 8, 16, 32 kHz, so we could have used a single payload type 
with a timestamp in 32 kHz units. From there the encoder could just switch 
from narrowband to wideband *without* signalling anything. The decoder 
would always decode at 32 kHz anyway, but would adapt to the audio 
bandwidth dynamically.

Again, I'm not sure how it applies to SILK, but from what I've read, it 
seems like it would only be a matter of adding two signalling bits (four 
rates) in the codec bit-stream itself -- if they aren't there already.

Cheers,

	Jean-Marc