Subject: [BEHAVE] TURN Issue #1: Bandwidth Param

From: Jonathan Rosenberg <[email-masked]>
Date: Fri, 07 Dec 2007 14:15:50 -0500

Open issue #1 from my slides (discussed during the WG meeting and at the 
ad-hoc), is the Bandwidth param. The issue is, its ill-defined right 
now. If no one wants it, we should drop it. If someone wants it, we need 
to define it.

During the WG meeting, folks spoke out in favor of it. Apparently there 
is currently deployed usage of this parameter. It is not being used for 
capacity planning. Rather, it is being used to rate limit consumers so a 
customer doesn't abuse the TURN server. Folks running that deployment 
strongly believed that the mechanism was needed.

So, the proposal is to keep this parameter. I have reached out to folks 
involved in RSVP to help us reuse a precise definition of how to measure 
the bandwidth parameter, since this is not a new problem. The proposed 
semantics would be something like this:

Client MAY include bandwidth in a request. If present, it is a request 
from the client that the server forward traffic to and from the client 
such that the total in each direction will not be excess of this value. 
Server MAY include a value in a response, even if there was not one in a 
request. The meaning of the value in the response, is that the server 
agrees to forward traffic to/from the client so long as it is below the 
bandwidth value. Any traffic above that value may be dropped by the 
server. If there is no value in the response, the server will relay 
traffic at best effort. Furthermore, there is never an absolute 
guarantee that the server will be able to relay data even if the usage 
is below the bandwidth parameter, and clients MUST be prepared for 
packet loss. Of course, there could be packet loss outside of the TURN 
server anyway.

Please comment if you are happy (or not) with keeping bandwidth with 
these proposed semantics.

Thanks,
Jonathan R.
-- 
Jonathan D. Rosenberg, Ph.D.                   499 Thornall St.
Cisco Fellow                                   Edison, NJ 08837
Cisco, Voice Technology Group
[email-masked]
http://www.jdrosen.net                         PHONE: (408) 902-3084
http://www.cisco.com

_______________________________________________
Behave mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/behave

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Fri, 07 Dec 2007 12:40:53 -0800

> these proposed semantics.

I am OK with keeping bandwidth, but I would like to add one thing.  The
problem with bandwidth is when TURN is used with ICE.  A TURN allocation
will be made but it will generally not be used excepted for connectivity
checks (i.e. far less than the bandwidth that will be used for video).
So most TURN servers used in an ICE environment will either
oversubscribe or waste resources.  Now if the TURN server knew the
*intent* of an allocation, it could have used it and statistics
collected from previous allocation's bandwidth usage to keep a good
balance between oversubscription and resource wasting.  So I am
proposing to have an additional attribute sent by the client in the
allocation that would tell the server if the allocation is meant for
ICE, or for another usage of TURN.

Thanks.

-- 
Marc Petit-Huguenin           [                                 ]
Home: [email-masked] [RFC1855-compliant space for rent ]
Work: [email-masked]            [                                 ]
[                                                               ]

---

From: Rémi Denis-Courmont <[email-masked]>
Date: Mon, 10 Dec 2007 10:53:16 +0200

Le Friday 07 December 2007 21:15:50 ext Jonathan Rosenberg, vous avez écrit :
> bandwidth value.

Hmm. I kinda doubt that a client which does not specify bandwidth would make 
any use of a returned bandwidth parameter.

I would rather go for the "let the network fix it"/best-effort approach that 
was proposed, at least if the client did not specify bandwidth. In practice, 
there are several reasons why bandwidth informations will often (if not 
always) be unavailable to the TURN client stack:
- file transfer or anything that wants to occur as fast as possible,
- among the many layers of software code between the TURN client and the 
codecs, it is likely the bandwidth info will be lost,
- ICE or other "RTP enhancements" may screw up the computation,
- in SIP+ICE caller case, you need to make TURN the allocation before you send 
the INVITE, which is obviously before you receive the SIP answer, which is 
when you (usually) decide which codec/bandwidth you're actually going to 
need.

> these proposed semantics.

I am fine with keeping the bandwidth attribute, but it definitely must be 
optional.

-- 
Rémi Denis-Courmont

---

From: Bob Andreasen <[email-masked]>
Date: Mon, 10 Dec 2007 10:29:58 -0500

> any use of a returned bandwidth parameter.

I agree that a client which does not request a bandwidth would
probably not use it -- but it is still very valuable to know if/when a
cap will be enforced.  One could adjust their codecs and bandwidth
given that information. 

> when you (usually) decide which codec/bandwidth you're actually going to

I believe your last point suggests adding the ability to adjust the
bandwidth (likely downward) via an allocation refresh.  Interesting,
but not a requirement in my opinion.  We are using bandwidth for rate
limiting not capacity planning.

> I am fine with keeping the bandwidth attribute, but it definitely must be

AOL (Bob Andreasen and Elena Furman) supports this.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Wed, 12 Dec 2007 23:30:02 -0500

inline:

Marc Petit-Huguenin wrote:
> oversubscribe or waste resources. 

That wouldn't be a problem, normally.

So when a client uses ICE it would do the allocation and provide the 
bandwidth that would be used if TURN is selected. If TURN is not 
selected, the client will, after ICE completes, drop the allocation. 
Thus, at any point in time, the allocations on the server will be all of 
those that are actually using TURN, plus a small number (those that have 
just arrived and for which TURN has not yet completed) that are only so 
far doing ICE. Law of large numbers would argue that for any reasonable 
server capacity, the overall capacity is dominated by the in-use 
allocations and not the transient ones.

-Jonathan R.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Wed, 12 Dec 2007 23:32:32 -0500

> cap will be enforced.  One could adjust their codecs and bandwidth

I think the main benefit in this case is really troubleshooting; if the 
client is seeing packet loss it can have a hint at whether the problem 
is that the traffic it is receiving is above its fair share, or because 
there is IP network packet loss.

But the point is, if a server is not doing per-user rate limiting, it 
would simply omit the bandwidth parameter in the response (assuming 
there was none in the request) and then it works as you say - best effort.

-Jonathan R.

---

From: Lars Eggert <[email-masked]>
Date: Thu, 13 Dec 2007 14:46:36 +0200

On 2007-12-7, at 21:15, ext Jonathan Rosenberg wrote:
> Client MAY include bandwidth in a request.

What exactly is bandwidth defined as? This is not trivial; see draft- 
ietf-ippm-bw-capacity. We do need a crisp definition if this is to work.

> If present, it is a request from the client that the server forward  
> traffic to and from the client such that the total in each direction  
> will not be excess of this value.

Do I read this correctly as a "please rate-limit me to X" option? What  
incentive does a client have to send this?

> Server MAY include a value in a response, even if there was not one  
> in a request. The meaning of the value in the response, is that the  
> server agrees to forward traffic to/from the client so long as it is  
> below the bandwidth value. Any traffic above that value may be  
> dropped by the server.

But this doesn't tell the client all that much. Yes, if it stays below  
the limit, it can be sure that the server doesn't rate-limit it, but  
congestion on the path can happen before that rate is reached, i.e.,  
the client can't depend on it. And because the server may forward  
traffic that exceeds this rate, the client has every incentive to see  
if it can send more than the signaled rate.

> If there is no value in the response, the server will relay traffic  
> at best effort. Furthermore, there is never an absolute guarantee  
> that the server will be able to relay data even if the usage is  
> below the bandwidth parameter, and clients MUST be prepared for  
> packet loss. Of course, there could be packet loss outside of the  
> TURN server anyway.

Right, this pretty much sums up my concerns.

TURN isn't a QoS-signaling solution, so I'm all for removing this  
option.

Lars (as an individual)

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Thu, 13 Dec 2007 07:59:08 -0800

> allocations and not the transient ones.

Yes, but it works only if the client drops the allocation and we do not
have the deployment experience to know how the ICE endpoints will
implement this.  This can be added as an extension later if needed or a
TURN server can collect statistics and guess the intent by inspecting
the application data forwarded.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Thu, 13 Dec 2007 23:59:49 -0500

Right - I was assuming that the client drops the allocation if it 
doesn't use it.

If you want to assume non-cooperative clients (ones that don't drop 
their allocations when they don't use them), bandwidth anyway won't be 
useful. Clients might decide to just not use the 'this is actually ice' 
flag, since they have no incentive to do so, and you're right back in 
the same boat.

-JOnathan R.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Fri, 14 Dec 2007 00:05:47 -0500

Lars Eggert wrote:
> What exactly is bandwidth defined as? This is not trivial; see 
> draft-ietf-ippm-bw-capacity. We do need a crisp definition if this is to 
> work.

Right - the plan was to take a leaky bucket definition from RSVP.

> incentive does a client have to send this?

The presumption is that the server has a relationship with the client 
and is there to support a particular application (say, voip). And so, if 
the client includes the value the server is likely to honor it and not 
drop data below that rate. If the client includes nothing the server 
might choose to rate limit at a value that is below what the client 
actually needs. So the client is incented to include it.

> 
>> Server MAY include a value in a response, even if there was not one in 
>> a request. The meaning of the value in the response, is that the 
>> below the bandwidth value. Any traffic above that value may be dropped 
>> by the server.
> 
> congestion on the path can happen before that rate is reached, i.e., the 
> client can't depend on it. And because the server may forward traffic 
> that exceeds this rate, the client has every incentive to see if it can 
> send more than the signaled rate.

This parameter is really only primarily useful for fixed rate 
applications for voip. For variable rate I agree its largely useless.

> 
>> If there is no value in the response, the server will relay traffic at 
>> best effort. Furthermore, there is never an absolute guarantee that 
>> the server will be able to relay data even if the usage is below the 
>> bandwidth parameter, and clients MUST be prepared for packet loss. Of 
>> course, there could be packet loss outside of the TURN server anyway.
> 
> TURN isn't a QoS-signaling solution, so I'm all for removing this option.

No, but TURN servers do provide a service (media relay of traffic) and 
it seems sensible for clients to ask for a particular level of service 
and be told if they get it or not.

-Jonathan R.

---

From: Lars Eggert <[email-masked]>
Date: Fri, 14 Dec 2007 17:02:09 +0200

Hi,

On 2007-12-14, at 7:05, ext Jonathan Rosenberg wrote:
> The presumption is that the server has a relationship with the  
> client and is there to support a particular application (say, voip).  
> And so, if the client includes the value the server is likely to  
> honor it and not drop data below that rate. If the client includes  
> nothing the server might choose to rate limit at a value that is  
> below what the client actually needs. So the client is incented to  
> include it.

I think my disconnect has been that I don't view the TURN server as an  
entity that enforces any sort of transmission behavior or capacity  
use. A TURN server should be a packet relay, plain and simple. It's  
basically a router that routes between a bunch of virtual TCP- or UDP- 
encapsulation interfaces and the Internet. TURN provides management  
for these virtual interfaces as well as data encapsulation. Nothing  
more, nothing less.

If specific deployment cases require a QoS enforcement point on such a  
router, we have a number of existing protocols that should be used to  
control this function, instead incorporating this functionality into  
TURN.

>> But this doesn't tell the client all that much. Yes, if it stays  
>> below the limit, it can be sure that the server doesn't rate-limit  
>> it, but congestion on the path can happen before that rate is  
>> reached, i.e., the client can't depend on it. And because the  
>> server may forward traffic that exceeds this rate, the client has  
>> every incentive to see if it can send more than the signaled rate.
>

But even for CBR it doesn't provide any guarantees, because congestion  
can happen before the signaled capacity is reached. (Unless TURN ties  
in with a "real" QoS mechanism, in which case why isn't that protocol  
used to manage QoS instead of TURN?)

> No, but TURN servers do provide a service (media relay of traffic)  
> and it seems sensible for clients to ask for a particular level of  
> service and be told if they get it or not.

TURN relays IP packets. Nothing in IP lets clients ask for a  
particular level of service. TURN can only be used to turn some knobs  
on the TURN server, which are unlikely to have much effect for the end- 
to-end path that the client flow is traversing.

Lars

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Fri, 14 Dec 2007 17:47:27 -0800

> flag, since they have no incentive to do so, and you're right back in

You are right.

---

From: Tim Moore <[email-masked]>
Date: Fri, 14 Dec 2007 11:07:43 -0800

> -----Original Message-----
> From: Lars Eggert [mailto:[email-masked]]
> Sent: Friday, December 14, 2007 7:02 AM
> To: ext Jonathan Rosenberg
> Cc: Behave WG
> Subject: Re: [BEHAVE] TURN Issue #1: Bandwidth Param
>
> Hi,
>

It doesn't provide guarantees but its still useful. The server bandwidth is not used as a "guarantee" of the rate that the server will support but its used as an upper limit that the server may support. Admins can decide that they are willing to reduce the quality of calls via the TURN server to increase the number of users a TURN server can support. The client wants to look at the server bandwidth number so its congestion algorithm can optimize the quality for this case.
An example would be a client that can send video at up to 2Mbps, a TURN server if it supports 1Gbps line rate only supports 500 connections at this speed, if you want to support more the server will need to rate limit the traffic. The server can do this without telling the client since the client will have congestion control to handle losses. However, if the client doesn't know about the server limit then it will want to try to increase its throughput to 2Mbps to provide the best quality. Every time it hits the server rate limit it will cause a packet lost which causes a quality drop. If it knows about the limit it can stay under the limit. It will still need to handle other congestion issues but the number of packet losses will be lower if the TURN server is the main bottleneck. This will mean the quality will be higher since the client can now stay closer to the server limit without causing the server to drop packets.

---

From: Philip Matthews <[email-masked]>
Date: Mon, 17 Dec 2007 20:13:21 -0500

I am thinking that it might be cleaner to move the BANDWIDTH  
attribute from the Allocate transaction to the ChannelBind  
transaction (assuming EKR's channel allocation proposal). This would  
have the following benefits:
* The client can wait to specify the bandwidth until after the Offer/ 
Answer exchange, when it knows the codec and also knows how many  
peers it will have.
* The client can specify the bandwidth to each peer separately, which  
is especially helpful under the long-lived allocation model.
* Marc's concern could be addressed by allocating some small amount  
of bandwidth to peers which have not yet been assigned a channel  
number, enough to do ICE connectivity checks.

Comments?

- Philip

---

From: Benny Prijono <[email-masked]>
Date: Tue, 18 Dec 2007 04:09:40 +0000

Philip Matthews wrote:
> I am thinking that it might be cleaner to move the BANDWIDTH attribute 
> from the Allocate transaction to the ChannelBind transaction (assuming 
> EKR's channel allocation proposal). This would have the following benefits:
> * The client can wait to specify the bandwidth until after the 
> Offer/Answer exchange, when it knows the codec and also knows how many 
> peers it will have.
> * The client can specify the bandwidth to each peer separately, which is 
> especially helpful under the long-lived allocation model.

The same can be achieved with Allocate/Refresh model as well. For
long-lived allocation, client can send Refresh with maximum
bandwidth before sending offer, then updating BANDWIDTH with the
actual bandwidth after offer/answer, and then setting BANDWIDTH to
minimum when the call has been cleared.

The benefit of this approach over ChannelBind with BANDWIDTH are:
  - bandwidth can be updated at any time. Currently we don't have
    ChannelUnBind and mechanism to refreshing ChannelBind, so with
    ChannelBind model, once bandwidth is requested it cannot be
    changed.
  - it works for peers that have not been assigned channel number
  - it requires less transaction to update the bandwidth (one for
    the whole media session, not one for each channel)

> * Marc's concern could be addressed by allocating some small amount of 
> bandwidth to peers which have not yet been assigned a channel number, 
> enough to do ICE connectivity checks.

How could you do that if you don't want to assign channel number for
the peer?

By the way, an open issue: should BANDWIDTH=0 be allowed?
Allowing BANDWIDTH=0 could help with long lived allocation (client
sends BANDWIDTH=0 when the call is idle). It looks like the spec
currently doesn't disallow it, so perhaps this could be allowed
explicitly. There should be no special semantic for BANDWIDTH=0,
i.e. it does not change both the allocation, permission, and channel
timers.

cheers,
  -benny

---

From: Philip Matthews <[email-masked]>
Date: Tue, 18 Dec 2007 02:07:02 -0500

On 17-Dec-07, at 23:09 , Benny Prijono wrote:

>> transaction (assuming EKR's channel allocation proposal). This  
>> would have the following benefits:
>> Offer/Answer exchange, when it knows the codec and also knows how  
>> many peers it will have.
>> * The client can specify the bandwidth to each peer separately,  
>> which is especially helpful under the long-lived allocation model.
>

Good point.

>  - it works for peers that have not been assigned channel number

Err.. If it hasn't been assigned a channel number, then that  
presumably means it is not important.

>    the whole media session, not one for each channel)

But presumably one updates the bandwidth every time a peer gets  
added, so I don't really see this.

>
>
>> * Marc's concern could be addressed by allocating some small  
>> amount of bandwidth to peers which have not yet been assigned a  
>> channel number, enough to do ICE connectivity checks.
>

The idea is that, if it is not worthwhile assigning a peer a channel  
number, then that peer is not (yet) important and doesn't deserve  
much bandwidth. With ICE, it only becomes important once it gets  
selected to carry the media stream. At that point, it gets assigned a  
channel and a bandwidth.

Never-the-less, I think the point about no way to update the  
bandwidth on a channel is a good one. I think this would have to be  
fixed to go with my idea.

> i.e. it does not change both the allocation, permission, and channel

Don't see why is should not be allowed.

On a different issue, I am trying to understand how to work with  
applications that do not have an obvious bandwidth value. A file  
transfer or a control channel, for example. I can see how TURN  
servers that work with streaming media might require and enforce the  
BANDWIDTH parameter, and other TURN servers that work as relays for  
non-streaming-media might just ignore the BANDWIDTH parameter, but it  
there a reasonable way to mix the two types of traffic on the same  
TURN server? I am still struggling with that one...

- Philip

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 18 Dec 2007 18:59:58 -0500

Philip,

I really don't see the point in starting another redesign on things to 
accommodate this long-lived allocation model. The use cases for this 
have been dubious at best, and the existing model supports them fine 
enough anyway. I just don't see why we should optimize or try to improve 
things significantly for these cases.

-Jonathan R.

Philip Matthews wrote:
> Err.. If it hasn't been assigned a channel number, then that presumably 
> means it is not important.
> 
> But presumably one updates the bandwidth every time a peer gets added, 
> so I don't really see this.
> 
> 
>>
>>
> number, then that peer is not (yet) important and doesn't deserve much 
> bandwidth. With ICE, it only becomes important once it gets selected to 
> carry the media stream. At that point, it gets assigned a channel and a 
> bandwidth.
> 
> 
> Never-the-less, I think the point about no way to update the bandwidth 
> on a channel is a good one. I think this would have to be fixed to go 
> with my idea.
> 
>>
> transfer or a control channel, for example. I can see how TURN servers 
> that work with streaming media might require and enforce the BANDWIDTH 
> parameter, and other TURN servers that work as relays for 
> there a reasonable way to mix the two types of traffic on the same TURN 
> server? I am still struggling with that one...
> 
> - Philip
> 
> 
>